juno-acid-state package

[axman6]

It seems that Juno and acid-state (Hackage) is a match made in heaven - eventual consistency with ACID guarantees on each of the nodes, allowing for any* haskell datatype to be easily turned into a distributed shared variable. I was planning to implement this myself, but since large changes are planned, I thought I'd make the suggestion so it can be implemented internally instead of needing to be rewritten once the changes are made.

[Ericson2314]

https://github.com/dmbarbour/haskell-vcache is also worthy of a mention then.

[axman6]

Agreed, that's probably an even better choice

[Ericson2314]

Also I opened https://github.com/dmbarbour/haskell-vcache/issues/5 It would be very nice to have vcache for persistence, something like IPFS for transport, and then this as a rather-smaller layer on top.

[axman6]

I should probably add that the reason I feel acid state is a good match for Juno is that it is basically implemented as a log of modifications to the datatype, with snapshots being stored occasionally, so the raft log perfectly marries with it.

[Ericson2314]

So the key is only so much of the data will change every iteration.

Initially, the Haskell data type / virtual heap is stored and transferred as a Merkle dag, and Raft only worries about a single changing pointer into the virtual heap. So top level mutable references and immutable everything else.

Then there is what VCache does with the PVars---they don't have to be top level. That corresponds to immutable nodes including black box names. The raft log would correspond to a single top-level root pointer, but rather transactions over the (also changing as variables are added or dropped) map of bindings.

[buckie]

FYI: updates are out.

I am interested in this idea but don't have the bandwidth to focus on it in the near term. To get me thinking about this correctly:

• what would the use case of this be?
• what would this give you over an integrated persistence layer (e.g. postgres) that you read from instead?

[Ericson2314]

I don't have a specific use-case in mind.

Versus postgres I have nothing super concrete either. The Merkle DAG maps to immutable functional program heap very well. But while relational DBs can probably the same Haskell datatypes, the mapping isn't as clean---some relational features aren't used, yet properties (immutability and access patterns) aren't expressible. Basically, I think some IPFS-like transport and persistence layer is the "correct" abstraction on which to layer this sort of consensus infrastructure, and I hope doing so long term allows more optimizations / less complexity.