improve JSON rendering

Ticket

+commit
 

In addition to rendering in Html, we want to format cards and their included cards as structured objects presented in JSON.

Key use cases:

(above tools recommended here)

 

The first major chunk of work (creating the object_renderer) is largely done.  

 

The biggest design needs:

  1. what does the data look like?
  2. how do we handle custom formatting?

We're putting ideas for the data in +examples below.

 

As for custom formatting, there are at least three options:

  1. Client side
  2. Server side 1: custom views
  3. Server side 2: custom actions

For client side, I think we should have very strong default options so that folks can get all the data that they need for cases where the client will be able to do significant data manipulation.

 

That said, there will also be cases where it really should come out cleanly right from Wagn, and I would think that option #2 is the right way to go.  Views are for all different ways to see the same data.  Actions are for data events that trigger some internal or external state change.  There may be some such triggers that involve JSON, but I would think that any server-side arrangement of data should happen in the views.

 

 

Sample output:

{
  "url": "http://cutch:3000/rules_to_export.json?pretty=true",
  "timestamp": "2013-09-23 15:37:11 -0600",
  "card": {
    "name": "rules_to_export",
    "type": "Search",
    "content": "{\r\n\"left\":{\"type\":\"Set\"},\r\n\"right\":{\r\n  \"type\":\"Setting\",\r\n  \"not\":{\"name\":[\"in\",\"*create\",\"*read\",\"*update\",\"*delete\"]}\r\n},\r\n\"edited_by\":{\"not\":\"WagnBot\"},\r\n\"return\":\"name\"\r\n}",
    "value": [
      {
        "name": "item type+*right+*option label",
        "type": "Phrase",
        "content": ""
      },
      {
        "name": "item type+*right+*default",
        "type": "Pointer",
        "content": "Image"
      },....

 

In this proposal, "content" is the raw content, where "value" involves processed content.  The above example shows search results.  Typical inclusions might be handled as follows:

...
  :content => "I include {{whatever else}}",
  :value=> [
    'I include:', 
     {
       :name => 'whatever else',
       :type => 'Phrase'
     },
     '!'
  ] 

 

note that there really isn't much need to keep labeling things as a "card".  Every hash is a card.

 

Preliminary Notes from a Demo/Example with Jquery Widgets

These widgets are somewhat rigid about how they they will take the json data, but you can re-arrange things in the success callback.  All the same, we can produce something closer if needed:

{data:"TheCardname":attr:{field1:value1,field2,value2...},children:{data:'PlusName',attr:...}}

 

 


I think I'm ready to close this ticket. Here's a look at what we can do now. I'm sure we can improve things, but I think we need to accrue more use cases to discuss further improvements (perhaps organized in a blueprint once we have enough to warrant that?)

 

For now we're putting out some pretty valuable output and it's easy to write mods when enhancements are needed.

--Ethan McCutchen.....2014-01-10 17:49:34 +0000

I'm fine with where this has gotten us so far, but it is missing the fundamental thing that I want/need. That is, content that is parsed and structured as JSON, not as a flat string. What we have is great for export, not so great for the other uses I have in mind.

 

I want similar for XML output. With XML output, we could actually replace the HTML rendering with XSLT based stylesheets. If you're not familiar with that idea, it may be work talking about sometime. With XML, the structure is really inline since we can just have elements within cards representing included cards. I already have to code to parse this kind of XML into updates to the main card and embedded cards. It does depend on having the original inclusion syntax in attributes for each card, but that is pretty straightforward.

 

The basic equation here is simple conversion between environments. Rendering data that javascript or XLST styling can display and edit, and then send back up to the server for writing updates. Originally I was thinking more in terms of doing this as XML because the inlining is more natural in a way, but now in working on some kind of live update editor the need it different. I need a structured representation (JSON is easiest given current support) that I can register with ShareJS. ShareJS events are then listened for and I update what the user sees in the browser. I can either save from the shared (structured) representation or what is in the browser.

 

From a modularity standpoint, I think doing this in the right way can move a lot of potential for customization out of Wagn view code and into Javascript or XSLT. This could vastly simplify rich_html views, which is what we wan't (I'm pretty sure, at least).

--Gerry Gleason.....2014-02-19 20:17:02 +0000

This probably belongs on a different card, maybe a new one. I think we need/want some of it for 2.0, but you know better. I don't want to add things that make getting to 2.0 harder, but we don't want to have to make any big changes to API things after that either.

--Gerry Gleason.....2014-02-19 20:20:00 +0000

You're right; comments added to a closed ticket are likely to get lost.

 

Our json doesn't get rendered to a string until "show", which is the method our controller uses to talk to format objects. Not a deep problem. But not one to be solved by hacking the controllers we've worked so hard to clean. If/when there's a real design put forward for dealing with object transfers, we'll talk then.

 

I don't see this as 2.0 priority stuff. It's not connected to any sort of strategy afaik.

--Ethan McCutchen.....2014-02-19 21:21:52 +0000

I'm not really arguing with the 'not for 2.0', except that it isn't that unreasonable to consider the deeper implications of my comment above. The 2.0 potential is more about the idea of rendering to a simplified XML and implementing the card wrapping, framing, and menus in XSLT and javascript. I'm down with wrapping up 2.0, but maybe 2.1 or 2.2? I'm just looking for some level of support, and then I might attempt a spike if I have some time or inclination.

--Gerry Gleason.....2014-02-20 03:21:45 +0000

There are lots of different issues at play here. There is the very standard notion of flexible rendering of json and xml as text and then expecting it to be parsed again on the other side, which is generally very easy to do in modules. I've already done it with the use cases listed on this ticket; there was no challenge using json for jstree or export. I haven't worked as much on the XML front with Wagn, but the same principles apply; if you're wanting to work with XSLT (which I did a bunch of about 8-9 years ago), then isn't the first step to work on rendering XML as flat text as is conventional?

 

Then you're proposing a much more unusual set of services involving transferring objects as objects and handling a variety of requests and such. None of that is connected to important use cases or strategy or anything as far as I've yet been able to understand.

 

I'm personally only interested in working on stuff that is oriented towards helping Wagn get traction. It's already exciting technology; what it needs is accessibility, both to mod developers and wagneers. This all just reads like a geek out to me. Not that there's anything wrong with geeking out, but that's what mods are for, right?

 

But if you really want to push this, can you start an idea somewhere?

 

Re the spike, it's always my preference for us to be able to articulate user stories and design first. Particularly as the community grows, it will be very helpful if you can work on articulating user stories that clearly relate the value of what you want to build.

--Ethan McCutchen.....2014-02-20 04:03:49 +0000