improve JSON rendering

For client side, I think we should have very strong default options so that folks can get all the data that they need for cases where the client will be able to do significant data manipulation.

That said, there will also be cases where it really should come out cleanly right from Wagn, and I would think that option #2 is the right way to go. Views are for all different ways to see the same data. Actions are for data events that trigger some internal or external state change. There may be some such triggers that involve JSON, but I would think that any server-side arrangement of data should happen in the views.

improve JSON rendering+example

The following are Ruby object representations that can be serialized with .to_json. We can also filter this structure with .as_json methods on the components of this structure before handing it off to serialization in .to_json.

Proposal #1

{:card =>   # I think {:cards => [ {:name=>"Home" ...},{} ...]} is what we want, although we can consider that different view parameters 
            # could select either/both kinds of root node. If :card represents a codename, we could put the cardname of that card as the
            # element name to give wagneer control ...
 { :attributes =>
    {:name =>"Home",
     :cardId =>80,    # optional, but when present is an unchangable Id of a real card
     :type_id =>3,   # :type would be the typename, I think, and we can have :typecode optionally as well
     :class=> ..., :style =>nil, :home_view =>nil, :item =>nil, :base =>nil  # These Html orientled style attributes may be supplied (when non-nill,
         # of course) as needed to  populate these attributes in the DOM, but most of the time we will let the Html rendering populate this style information 
          },
   :result =>[  #  :result is probably the wrong name, these are the child nodes, the woudl be inside the <card> </card> tags in Xml, and 
                #  in Json this is often called :children.  I think :contents would make the most sense for cards, but maybe that is a
                #  codename and changable by rename?
      "<h1>Welcome to Wagn!</h1>\n<p>&nbsp;</p>\n<p>",
      {:cardlink =>{   # view controls on the nested inclusions would say whether this is rendered like this as a nested data object represeting the
                       # link or something rendered at a link to string
          :class =>"known-card",
          :url =>"/Config"
          :text =>"Start configuring"
      }},
      " or learn more:</p>\n<ul>\n<li>",
      {:link =>{
          :class =>"external-link",
          :url =>"http://wagn.org/wagn/configuration",
          :text =>"Configuration"
      }},
      "</li>\n<li>",
      {:link =>{
          :class =>"external-link",
         :url =>"http://wagn.org/wagn/Documentation",
          :text =>"Other Documentation"
      }},
      "</li>\n<li>",
    {:link =>{
          :class =>"external-link",
          :url =>"http://wagn.org/wagn/Support",
          :text =>"Support"
      }},
      "</li>\n</ul>\n<div><em>Double-click content to edit.</em></div>\n<p>&nbsp;</p>"
   ]
 }
}

Proposal #2

{ 
  :card => {
    :name    => 'whatever',
    :type    => 'Basic',
    :content => (raw content), # for export
    :parsed_content=> [
       'I include:', {
           # should this be in :card?
           :name => 'whatever else',
           :type => 'Phrase'
        }, '!'
      ]
    } 
    # other card metadata
  }
  # request metadata, etc.
}

Proposal #3 (actually generated in wagn)

Sample output:

{
  "url": "http://cutch:3000/rules_to_export.json?pretty=true",
  "timestamp": "2013-09-23 15:37:11 -0600",
  "card": {
    "name": "rules_to_export",
    "type": "Search",
    "content": "{\r\n\"left\":{\"type\":\"Set\"},\r\n\"right\":{\r\n  \"type\":\"Setting\",\r\n  \"not\":{\"name\":[\"in\",\"*create\",\"*read\",\"*update\",\"*delete\"]}\r\n},\r\n\"edited_by\":{\"not\":\"WagnBot\"},\r\n\"return\":\"name\"\r\n}",
    "value": [
      {
        "name": "item type+*right+*option label",
        "type": "Phrase",
        "content": ""
      },
      {
        "name": "item type+*right+*default",
        "type": "Pointer",
        "content": "Image"
      },....

In this proposal, "content" is the raw content, where "value" involves processed content. The above example shows search results. Typical inclusions might be handled as follows:

...
  :content => "I include {{whatever else}}",
  :value=> [
    'I include:', 
     {
       :name => 'whatever else',
       :type => 'Phrase'
     },
     '!'
  ]

note that there really isn't much need to keep labeling things as a "card". Every hash is a card.

Preliminary Notes from a Demo/Example with Jquery Widgets

These widgets are somewhat rigid about how they they will take the json data, but you can re-arrange things in the success callback. All the same, we can produce something closer if needed:

{data:"TheCardname":attr:{field1:value1,field2,value2...},children:{data:'PlusName',attr:...}}

The main question is how to specify customized views. For Xml, I'm inclined to use a standardized set of elements for cards and links as implemented and allow for xslt based views (i.e. send supporting custom views as xslt scripts that function as stylesheets). This xslt (or similar) processing could also be server side, and we could use something like Rabl for a similar role in json (taking the generic objects and using the object names and attributes expected by the client (customized).

My sense is that the default xml/json data should be, essentially, a standard wagn export format. so including stuff like classes and such seem a bit much. I do think it would be nice to include an "origin" (or some similarly named) key whose value is the url for the card (html) from which the export originated.

I'm inclined to say the fields should predominantly use keys not ids, because any given import will be likely to use the same names or variants thereof, but the ids may need to be entirely different.

--Ethan McCutchen.....2012-09-28 21:27:43 +0000

I'm going to improve the examples. To outline what I'm planning, a couple of links I'm still exploring: JQuery plugins: jstree and

--Gerry Gleason.....2012-10-08 15:31:45 +0000

So, the comment is about row views. Like a line view (closed), but meant to be a row in a table/grid. In Json, I'd want to render a type with a template (*content rule) as named attributes to the data model rather than as child elements or an array, etc. In Html, we may use a table row format. Do we need a distinct templete, so a *row rule?

--Gerry Gleason.....2012-10-10 15:05:21 +0000

re row views, I have a proposal coming soon for that; I don't think we have to resort to new rules. (In fact, I think the solution will have a ton of other uses)

--Ethan McCutchen.....2013-03-11 22:53:13 +0000

re question 1, I added a stab above in Proposal 2. With Proposal 1 (which isn't terribly different), I'd want to see how the recursion actually works.

--Ethan McCutchen.....2013-03-12 03:23:26 +0000

This card is not showing up in Tickets and I did clear the cache. I just changed the type to Ticket (from Idea), that should work, right?

--Gerry Gleason.....2013-03-13 20:58:25 +0000

I'm not sure I'm following how you want to "see how the recursion works". Is that about clarifying the example, or describing how it would work?

--Gerry Gleason.....2013-03-13 21:51:15 +0000

Your Proposal 2 doesn't really show how it would handle multiple cards, like a deck or pack as a single object. Would you just have an Array of those? Seems reasonable, just asking.

--Gerry Gleason.....2013-03-13 21:53:07 +0000

Finally noticed the big difference here. My form (P1) is all parsed and the inclusions and links are objects. It isn't what we want for export, but it could be what we want for an API that is used by advanced javascript enhancements. The point is that I don't want to have to parse the inclusions in javascript, I want that on the server side.

Do you think that makes any sense for UX based use cases?

--Gerry Gleason.....2013-03-13 21:57:34 +0000

Lots of good questions. Some thoughts:

1. yes, this should be a ticket. If we keep the scope small, it might go 1.11, but 1.12 is the better bet.

2. by recursion, I would expect to see something more fractal. In Proposal 1 I don't see any patterns recurring further down the tree. In Proposal 2, you see :card and :view twice as an attempt to show how recursion might work. Basically, it probably just boils down to having some example inclusions in addition to links.

3. With the wagn web api, we often view many cards, but we always use one as the window. (Eg, you go to a named search card to see a list of cards returned by that card). That is what I would propose we do for this rep as well. So mulitiple cards would look something like {:card=>{:name=>'my search'}, :view=>{:parts=>[{(card1)},{(card2)},{(card3)},{(card4)}]}}

4. P2 had an attempt to show both parsed and unparsed results. In the use cases we listed export, which would often benefit from unparsed, particularly if exporting to another Wagn. I was using "content" (part of the card object) to refer to the unparsed portion and "parts" to refer to the parse part.

I really think P2 is just a slightly crisper version of P1. Really P1 v2 :). I like that it maps a little more cleanly to Wagn structures with fields like "view" and "content", and that it shaves off a little fat like "attributes", which seems redundant to me.

--Ethan McCutchen.....2013-03-14 02:46:39 +0000

I think I'm ready to close this ticket. Here's a look at what we can do now. I'm sure we can improve things, but I think we need to accrue more use cases to discuss further improvements (perhaps organized in a blueprint once we have enough to warrant that?)

For now we're putting out some pretty valuable output and it's easy to write mods when enhancements are needed.

--Ethan McCutchen.....2014-01-10 17:49:34 +0000

I'm fine with where this has gotten us so far, but it is missing the fundamental thing that I want/need. That is, content that is parsed and structured as JSON, not as a flat string. What we have is great for export, not so great for the other uses I have in mind.

I want similar for XML output. With XML output, we could actually replace the HTML rendering with XSLT based stylesheets. If you're not familiar with that idea, it may be work talking about sometime. With XML, the structure is really inline since we can just have elements within cards representing included cards. I already have to code to parse this kind of XML into updates to the main card and embedded cards. It does depend on having the original inclusion syntax in attributes for each card, but that is pretty straightforward.

The basic equation here is simple conversion between environments. Rendering data that javascript or XLST styling can display and edit, and then send back up to the server for writing updates. Originally I was thinking more in terms of doing this as XML because the inlining is more natural in a way, but now in working on some kind of live update editor the need it different. I need a structured representation (JSON is easiest given current support) that I can register with ShareJS. ShareJS events are then listened for and I update what the user sees in the browser. I can either save from the shared (structured) representation or what is in the browser.

From a modularity standpoint, I think doing this in the right way can move a lot of potential for customization out of Wagn view code and into Javascript or XSLT. This could vastly simplify rich_html views, which is what we wan't (I'm pretty sure, at least).

--Gerry Gleason.....2014-02-19 20:17:02 +0000

This probably belongs on a different card, maybe a new one. I think we need/want some of it for 2.0, but you know better. I don't want to add things that make getting to 2.0 harder, but we don't want to have to make any big changes to API things after that either.

--Gerry Gleason.....2014-02-19 20:20:00 +0000

You're right; comments added to a closed ticket are likely to get lost.

Our json doesn't get rendered to a string until "show", which is the method our controller uses to talk to format objects. Not a deep problem. But not one to be solved by hacking the controllers we've worked so hard to clean. If/when there's a real design put forward for dealing with object transfers, we'll talk then.

I don't see this as 2.0 priority stuff. It's not connected to any sort of strategy afaik.

--Ethan McCutchen.....2014-02-19 21:21:52 +0000

I'm not really arguing with the 'not for 2.0', except that it isn't that unreasonable to consider the deeper implications of my comment above. The 2.0 potential is more about the idea of rendering to a simplified XML and implementing the card wrapping, framing, and menus in XSLT and javascript. I'm down with wrapping up 2.0, but maybe 2.1 or 2.2? I'm just looking for some level of support, and then I might attempt a spike if I have some time or inclination.

--Gerry Gleason.....2014-02-20 03:21:45 +0000

There are lots of different issues at play here. There is the very standard notion of flexible rendering of json and xml as text and then expecting it to be parsed again on the other side, which is generally very easy to do in modules. I've already done it with the use cases listed on this ticket; there was no challenge using json for jstree or export. I haven't worked as much on the XML front with Wagn, but the same principles apply; if you're wanting to work with XSLT (which I did a bunch of about 8-9 years ago), then isn't the first step to work on rendering XML as flat text as is conventional?

Then you're proposing a much more unusual set of services involving transferring objects as objects and handling a variety of requests and such. None of that is connected to important use cases or strategy or anything as far as I've yet been able to understand.

I'm personally only interested in working on stuff that is oriented towards helping Wagn get traction. It's already exciting technology; what it needs is accessibility, both to mod developers and wagneers. This all just reads like a geek out to me. Not that there's anything wrong with geeking out, but that's what mods are for, right?

But if you really want to push this, can you start an idea somewhere?

Re the spike, it's always my preference for us to be able to articulate user stories and design first. Particularly as the community grows, it will be very helpful if you can work on articulating user stories that clearly relate the value of what you want to build.

--Ethan McCutchen.....2014-02-20 04:03:49 +0000

sharks

monkeys

platypuses

everyone

improve JSON rendering

Ticket