Immutability, Persistence and History+overview
This presentation makes a general case for these attributes of a data system. This thinking is from javascript work at Facebook as embodied by React, Flux or Reflux, but I think the underlying thinking about data system properties feels very closely related to core concepts of the card system implemented in wagn.
In the long run, we should be looking at these specific tools for the front end, but that can wait. Two archetectural concepts that have come up lately are more connected to how we should include these properties in the back end. At first I was thinking that making a clear split between the Card model and the history module was to allow us to skip all that work if we didn't need to refer to state before the current version. Now I'm thinking that the representation of the current state as the result of a flow of actions The cards and card_references tables can be re-generated if you have a store of the history. In fact, with the complete history, you can restore any state of the data, or compare them or ...
Immutability, Persistence and History+needs
To have changing a deck (some cards) and history abstracted from rendering and representing current state. Now that History is more of a module and change processes are all tied together and organized by the event system, I think the main idea here is to split rendering from a stream of changes represented as changesets. With events the code can be configured with the current History system as module, and/or a second system that encodes updates as a stream of changes. I'm going to leave the original +needs text below for now and try to evolve this idea into something more sensible.
I have been Slacking in other workspaces and brainstorming about federated content architectures and I came across a project called ValueFlows that is now defined on top of a W3 standard for ActivityPub (I can edit in the links later). that is used for a lot of new OS social media tools including Mastadon. The idea here would be to define the necessary ValueFlow specifications to transform Card change events into VF events. This can use something based on our YAML representation of card hands (subsets of a deck) for packaging cards in modules libraries and updates. There is an implementation based on JSON, but serialization formats can be fungable as needed for the context.
This wplit was also something to be considered for Decko instances that might experience extreme scaling and need to deliver data at Wikipedia rates. Having a read-only front end and caching all the renderings of that state. Systems engineering issues are better if the systems are designed as flows of events in a larger system, and the event system driving the AR based tables of actions, acts and changes IIRC that are used now to render (and edit) from revisions.
Old text below:
Instead implement a Store abstraction similar to the one used by Flux or Reflux, only instead of being a persistant store in a browser, it can use a storage model (active record to start). The key to this abstraction is that you never mutate a card directly, you send the update payload down the line in a change action to the Stores. Listener/observers to the actions and the resulting Store transformations signalled to downstream listeners trigger all the other changes and events that need to happen. Otherwise, the stores are 'get-only', you can only read data or push payloads at the store.
This would allow us to persistently cache objects. In a lot more places we would have to carry the equivalent of action_id for card_ids so that we don't have to break the cache when rules change. Any cached data whether in AR or a cache store would have the persistant and immutable properties. New data will have new action_ids to be cached or stored, but all these ids are internal to the store anyway. Memory and cache instances of an abstract card will be made concrete with an action_id (except when new?).
In case it isn't clear, this is more a thought experiment than anything at this point. It is interesting to think about where we use ids and where the need is to refer to any version vs. a specific version of a card. It is clear that the ids need to be assigned by the Store, and they are (by AR now). If it is necessary for card_ids and action_ids to be either 1) disjoint or 2) when there is a card_id and action_id that are the same (equal) the action is for that card. That might typically be the first action when the card is created, but it doesn't have to be.
WQL will be one of the harder parts to re-design in these terms, but I think the result could be very worthwhile. The parsing of queries probably doesn't change much, but the process of executing searches moves from being mostly a Card thing to being part of the Card Store. Anything related to the references table is also inside the Store (for support of queries), and similarly the cards table would be designed to support search by WQL as compiled into AR queries into Store tables.
It will also put the updating of references into the store. Card code won't need to know how references and history is represented in the Store, the Store just does what it needs to based on the actions in processes, and after that reads and searches will return the new data. The Store might be lazy about how much work it does at the point of commit, all it has to do is commit the actions to a transaction log (our history tables already are DB style roll-back/forward logs) and only update what's in the cache for a given card_id when the store is asked for the latest action on that card. The reference update queue only has be fully processes if you need to do a search now. You can be lazy about lots of stuff.
Immutability means that the Card objects have to obey the immutability contract. If you change it, it becomes a new object (which when stored/committed will get a new action_id), and mutators have to return new objects that you need to store locally (often overwriting a local variable that had the original one), but the more functional our code is, the more it just returns a new (vs. mutated) object up the function stack.
the needs section begins "Remove activerecord dependency from Card.". That's not a need, that's a step in an implementation plan. Can you connect this to needs?