Necessary conditions for robustness and portability

[spirobel]

To increase the robustness and portability of wallet3 it should be built around a purely functional library that meets the following conditions:

1. C ABI
2. No concurrency and networking mixed in.
3. No hidden assumptions about the environment.
4. No side effects and hidden control flows.
5. No unnecessary dependencies sprinkled in (for example boost)
6. Minimal amount of code paths per function.

One of the lessons to learn from wallet2:
instead of creating one very tightly integrated piece of software that tries to solve all of these things at once, the following issues should be dealt with by the wallet implementers:

• concurrency (respond to user input and not stall)
• networking (communicating with the daemon)
• parallelism (threading)
• logging

The side effect of leaving these problems to implementers is that there will be a better separation of concerns. There should be no full implementation of a wallet that does these things in the monero core repo.

We need to avoid at all costs that the example wallet library is turned again into the de facto standard wallet implementation.

We should make a hard rule that no code gets checked into the main monero repo that breaks these rules:

1. "example" wallet code that touches the concerns mentioned above
2. code that breaks the 6 conditions necessary for robust and portable wallet code.

I invite you to read the custom vendored easylogging_c++ library that gets shipped with Monero:
https://github.com/monero-project/monero/blob/master/external/easylogging%2B%2B/easylogging%2B%2B.cc

This library introduces a dependency on signal.h even for code that should be strictly functional and without side effects.
This is an example for 3 issues that we need to avoid:

1. Code with side effects that serve no purpose
2. Dubious unnecessary dependencies
3. Creeping hidden assumptions about the environment the code is running in

Ask yourself this: do you want the code that handles your money contain a 4000 line library to print strings to a console?
(in many different colors and formats)

[rbrunner7]

We need to avoid at all costs that the example wallet library is turned again into the de facto standard wallet implementation

If this is an important center argument of your issue, I have to say I don't understand it, and I don't see how that would follow from the rest of what you write.

What is, at its core, the big problem you see with a "de facto standard wallet implementation"? A problem so big and so grave that we have to avoid it "at all cost"?

This workgroup here currently has the express top-level goal of implementing exactly that, a standard wallet implementation that most wallet apps can use, at least as I currently understand it, so seems to me your argument can in theory have very wide repercussions.

By the way, did you already look closer at @UkoeHB 's Seraphis library? I am not sure, but it could be that library is already in large parts what you wish for with your points 1. to 6.

[sausagenoods]

I also agree that wallet3 should implement minimal wallet functions. A higher level program should deal with logging, parallelism and concurrency.

[spirobel]

I also agree that wallet3 should implement minimal wallet functions. A higher level program should deal with logging, parallelism and concurrency.

Great to see more people share this view!

This workgroup here currently has the express top-level goal of implementing exactly that, a standard wallet implementation that most wallet apps can use,

you are repeating the same mistake again. Just provide a library for all the cryptography with a C ABI. Keep it purely functional. The definition of the API and the implementation of the wallets can not be done by the same person.
Otherwise everything will be entangled.

By the way, did you already look closer at @UkoeHB 's Seraphis library? I am not sure, but it could be that library is already in large parts what you wish for with your points 1. to 6.

Just to clarify, you mean this code, right? https://github.com/UkoeHB/monero/tree/seraphis_lib/src/seraphis
It fails the requirements. It does not provide a C ABI. It has boost dependencies. Again: this already looks very much like wallet2.
Create an API and explain it to others. Don't try to do everything as a single person. Otherwise everything will be entangled.
We should have multiple teams working on this instead of just letting one person create a giant monolith that is then dependent on this one person.

[rbrunner7]

you are repeating the same mistake again

You can of course just claim that something is a mistake without giving arguments that other people can read, judge, form their own opinion on, and comment, but I am pretty sure that this will not be very successful. If you claim this workgroup (not merely I myself, I am not alone, we are a workgroup) commits a mistake, please state what the mistake is in your opinion, and most importantly argue why it's a mistake.

It fails the requirements. It does not provide a C ABI.

Monero is implemented in C++. Thus also @UkoeHB 's Seraphis library, because it's meant to become an integral part of the Monero codebase, is written in C++. Providing a C ABI would be a fundamental change of direction. Without very good arguments why that Seraphis library should be rewritten to offer a C ABI you won't get anywhere IMHO. Just claiming "it fails the requirements" doesn't cut it.

[spirobel]

Providing a C ABI would be a fundamental change of direction.

No it would not be. You can wrap the functions that you want to expose as an API as extern "C".

The cryptographic purely functional components can also be written as pure C. That would definitively not be "a fundamental change in direction".

without giving arguments

just to make sure we are on the same page:
From you point of view, what are the biggest problems with wallet2.cpp that we need to avoid repeating?

[rbrunner7]

From you point of view, what are the biggest problems with wallet2.cpp that we need to avoid repeating?

I am ready to discuss that anytime, but I think this issue is not the best place to do so. You could ask this question in the Matrix room or the bridged IRC channel, where I think other people would be interested to read my answer and the discussion that may result from it.

You opened a fresh issue here, with proposals what this workgroup should build, and how it should build it, and without good arguments for what I see as quite extraordinary proposals I fail to see the value of this particular issue here.

The cryptographic purely functional components can also be written as pure C

Why should they? Convince me. No need to argue all over the place, for starters we could very well concentrate on this: You seem to put high value on having a pure C interface. Please elaborate.

[UkoeHB]

Monero development has never exposed a C ABI. Even src/crypto.h wraps the core crypto which is in C into a C++ namespace.

[spirobel]

Please elaborate.

Here are three user stories: #2 I don't see how wallet3 in its current form addresses those.

Matrix room or the bridged IRC channel,

Please open another issue here and answer this question. You love to ask others to elaborate. So please do so too. I am unsure if you are aware of the many short comings of wallet2. This question is highly relevant for making sure the quality and UX of Monero software improves. We need to have a discussion about this and we can't let this get buried in some IRC chat log.

Convince me.

What is your official title with this workgroup by the way?

[sneurlax]

One argument for exposing a C ABI is so that other languages like Go and Rust can access wallet fundamentals more easily.

But Monero is responsible for being complete as a standalone project.

Those developers that need features which benefit other languages ought to take it upon themselves to change the code to play with Go, Rust, Dart, etc., more nicely. Monero is in C++ and monero core developers develop in C++. You'll need significant traction/consensus in order to change the development path (which was set over a year ago now and after papers and publications were shared and comments requested) like this. My recommendation is that if you want these changes, start contributing them now, because I don't think the core developers will do this work for you unless there's overwhelming demand

[sneurlax]

Maintaining a fork which is more portable would be a valuable contribution that would probably be merged in and used if it's the only or best way to implement the user stories from your other issue. I'd love to collaborate with you on this issue: I also like to access Monero functions with other languages I prefer more than C++. If you'd like help, let's try and make the code more portable, but my response with regards to this issue is just "let the core C++ developers continue, if you want other languages supported better too, let's work together on that now"

[UkoeHB]

Aside from 'C ABI' and 'no boost', the seraphis library already satisfies your requested behavior. In fact, you could build your very own wallet using the seraphis library :). At some point though, you run out of things that can be librarized and start needing to develop specific solutions.

[UkoeHB]

It fails the requirements. It does not provide a C ABI. It has boost dependencies. Again: this already looks very much like wallet2.
Create an API and explain it to others. Don't try to do everything as a single person. Otherwise everything will be entangled.
We should have multiple teams working on this instead of just letting one person create a giant monolith that is then dependent on this one person.

These comments are hard to believe. Have you looked at wallet2? Have you read the seraphis unit tests demoing the library? Do you have qualified C devs lined up to rewrite the library? Have you been following the seraphis research and development that has been going on for the last 15 months? It's hard to see your suggestions as realistic.

[spirobel]

In fact, you could build your very own wallet using the seraphis library :)

That would be very hard because nothing is documented at the moment. It is unclear which functions will form a stable API. And there is no C ABI so the interop from most languages is hard.

At some point though, you run out of things that can be librarized and start needing to develop specific solutions.

At that point you have to start documenting and reaching out to others so they can implement the rest. If you continue to implement the rest of the wallet, many assumptions about "what a wallet is" will be hardcoded in the final blob that is delivered after the waterfall development is over.

Do you have qualified C devs lined up to rewrite the library?

There is no need to rewrite everything in C. You can just wrap the functions you want to expose in extern "C" so others can use them as well. Also as you mentioned: many of the crypto functions are already written in C. Just expose them, so that people outside of the CPP world can use this too.

but my response with regards to this issue is just "let the core C++ developers continue, if you want other languages supported better too, let's work together on that now"

How can we work on that, if nothing gets documented and we just ought to accept a giant blob after the waterfall development is over? It is not a big problem to expose a C ABI instead of a CPP API. Wrap the functions in extern "C".

So please @UkoeHB lets talk about the API design. Where is the hurt in exposing a C ABI instead of a cpp API? It would help a lot of people immensely!

I'd love to collaborate with you on this issue: I also like to access Monero functions with other languages I prefer more than C++.

@sneurlax okay lets do this! The user stories are a great point to start!

[UkoeHB]

At that point you have to start documenting and reaching out to others so they can implement the rest. If you continue to implement the rest of the wallet, many assumptions about "what a wallet is" will be hardcoded in the final blob that is delivered after the waterfall development is over.

One of the advantages of being on IRC is you learn things like 'koe is winding down involvement with Monero and plans to cease development work soon'.

You can just wrap the functions you want to expose in extern "C" so others can use them as well.

There are plenty of C++ types that are also exposed. You are suggesting a pretty serious rewrite. I recommend watching the code walkthrough video.

Also as you mentioned: many of the crypto functions are already written in C. Just expose them, so that people outside of the CPP world can use this too.

Monero is NOT a crypto library, and absolutely should not expose crypto functions for general use.