Discussions

Expand all | Collapse all

Ruby support via Thrift

  • 1.  Ruby support via Thrift

    Posted 06-29-2017 14:39

    Anyone out there using MapD with Ruby (via Thrift)? I generated the Ruby classes from the mapd.thrift IDL file (thrift --gen rb mapd.thrift) and found that the “mapd_types.rb” file it generated has types with circular references, and Ruby just can’t handle it.

    2.3.3 :002 > require './app/models/map_d/mapd_types'
    NameError: uninitialized constant TDatum
    Did you mean?  TDatumVal
    	from app/models/map_d/mapd_types.rb:80:in `<class:TDatumVal>'
    

    If I change the order so that TDatum is defined first, we run into a similar problem where TDatumVal is now undefined. There may be other circular references in this file, but this is the first one I tripped on.

    Is this a solved problem, or are we breaking new ground here?



  • 2.  RE: Ruby support via Thrift

    Posted 06-29-2017 14:45

    This is something we’ve hit in the Python world as well. The issue is that we currently represent arrays as recursive types, which some languages don’t like.

    For Python I have a script at https://github.com/mapd/mapd-core/blob/master/SampleCode/fix_recursive_structs.py which circumvents the issue by delaying the initializing a few things until after all the types have been defined.

    Alternatively you can simply remove array support. Not desirable, but it will at least let you continue without further hacks:

    Modify mapd.thrift to remove these two lines:
    4: list<TDatum> arr_val from TDatumVal (~line 43)
    4: list<TColumn> arr_col from TColumnData (~line 81)

    and then regenerate the ruby bindings with thrift -gen ...



  • 3.  RE: Ruby support via Thrift

    Posted 06-29-2017 14:57

    Thanks for the quick reply. Are there plans to change how arrays are represented to avoid this problem, or are you sort of stuck with doing it this way?

    I ended up exploiting Ruby’s feature of re-opening classes to simply define “skeleton” constants before where they were being referenced, and then let the real definition come later. For example:

    # Hack to avoid circular reference
    class TDatum
    end
    
    class TDatumVal
      ....
      ARR_VAL => {:type => ::Thrift::Types::LIST, :name => 'arr_val', :element => {:type => ::Thrift::Types::STRUCT, :class => ::TDatum}}
      ....
    end
    
    class TDatum
      ....real definition....
    end
    

    Thanks again.



  • 4.  RE: Ruby support via Thrift

    Posted 06-29-2017 15:04

    One other question related to this. The MapD service in mapd.thrift has this:

      TSessionId connect(1: string user, 2: string passwd, 3: string dbname) throws (1: TMapDException e)
    

    How do I specify the hostname and port to connect to given this API definition? Is that embedded in dbname somehow?



  • 5.  RE: Ruby support via Thrift

    Posted 06-29-2017 15:55

    Ah, nevermind, I see that I have to create the thrift transport and protocol objects which define this stuff.

    Thrift newbie.