refactor(juno): integrate common interface with rawDB as trieDB engine

[MaksymMalicki]

1. This PR should be reviewed after refactor(state): remove DB writes from the new state to make writes atomic #3360

2. It integrates the common interfaces across the project, including in utilities and tests. This PR uses rawDB - direct pebbleDB reads/writes - as a trieDB engine. New components of the pathDB engine will be introduced in the follow-up PRs.

3. Some parts of the logic weren’t straightforward to unify due to the different ways the old and new states interact with PebbleDB. The old state is tightly coupled with IndexedBatch for reads and writes, while the new state is designed to read using raw DB/snapshots and write using regular PebbleDB Batch. As a result, the following methods were "duplicated" and tailored to the way the respective state and trie functionalities are meant to interact with the DB:

• blockchain.Store

• blockchain.GetReverseStateDiff

• blockchain.Finalise

The adapting had to be also done for the prove logic inside of rpc/v8/storage.go, rpc/v8/storage_test.go, rpc/v9/storage.go, rpc/v9/storage_test.go

4. The experimental --new-state boolean CLI flag is now fully integrated. It can be used via the command line and in the dbcmd utilities.

5. The new state package includes some minor bug fixes discovered during previous work (#2912) while debugging sync, RPC, and E2E tests.

6. A new command was added to the Makefile, to run the unit tests strictly with the new state and trie implementations in the future CI/CD:

test-new-state: clean-testcache rustdeps ## Run tests with new state
	USE_NEW_STATE=true go test $(GO_TAGS) -v ./...

[codecov]

Codecov Report

❌ Patch coverage is 35.49020% with 329 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.93%. Comparing base (beeb202) to head (2b58353).

Files with missing lines	Patch %	Lines
blockchain/blockchain.go	32.32%	123 Missing and 11 partials ⚠️
rpc/v9/storage.go	31.25%	62 Missing and 4 partials ⚠️
rpc/v8/storage.go	32.63%	60 Missing and 4 partials ⚠️
core/state/history.go	0.00%	9 Missing ⚠️
db/memory/db.go	0.00%	7 Missing ⚠️
db/pebble/db.go	0.00%	7 Missing ⚠️
db/pebblev2/db.go	0.00%	7 Missing ⚠️
db/remote/db.go	0.00%	6 Missing ⚠️
cmd/juno/dbcmd.go	60.00%	2 Missing and 2 partials ⚠️
core/state/state.go	33.33%	4 Missing ⚠️
... and 9 more

❌ Your patch check has failed because the patch coverage (35.49%) is below the target coverage (60.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

@@                    Coverage Diff                     @@
##           maksym/state-atomicity    #3187      +/-   ##
==========================================================
- Coverage                   75.53%   74.93%   -0.60%     
==========================================================
  Files                         384      385       +1     
  Lines                       34910    35284     +374     
==========================================================
+ Hits                        26368    26440      +72     
- Misses                       6684     6981     +297     
- Partials                     1858     1863       +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.