Skip to content

wip: Raw structure of improvements to gun that run in production#690

Open
speeddragon wants to merge 98 commits intoneo/edgefrom
impr/gun
Open

wip: Raw structure of improvements to gun that run in production#690
speeddragon wants to merge 98 commits intoneo/edgefrom
impr/gun

Conversation

@speeddragon
Copy link
Collaborator

@speeddragon speeddragon commented Feb 26, 2026

In this PR, there are a few improvements to how gun handles the connection.

  • Connections are fetched using ETS instead of relying on the gen_server.
  • Support for multiple connections for the same peer.
  • Separate them from read (GET/HEAD) and write (POST/PUT).
    • This was most useful when handling S3 read/write, not sure if very useful for this case.
    • This can be configured in a config file by setting conn_pool_read_size and conn_pool_write_size.
  • Improve and fix metrics for both gun and httpc clients.
    • Added category to separate duration metrics by endpoint (/tx, /chunk, etc).

TODO

  • Improve code structure.
  • Review TODO:

@speeddragon speeddragon force-pushed the impr/gun branch 2 times, most recently from acd6a2f to bed3351 Compare February 27, 2026 00:14
@speeddragon speeddragon changed the base branch from edge to neo/edge February 27, 2026 00:14
Comment on lines +22 to +25
setup_conn(Opts) ->
ConnPoolReadSize = hb_maps:get(conn_pool_read_size,Opts, ?DEFAULT_CONN_POOL_READ_SIZE),
ConnPoolWriteSize = hb_maps:get(conn_pool_write_size,Opts, ?DEFAULT_CONN_POOL_WRITE_SIZE),
persistent_term:put(?CONN_TERM, {ConnPoolReadSize, ConnPoolWriteSize}).
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was trying to find a better way to define this, but I couldn't.

@samcamwilliams samcamwilliams force-pushed the impr/gun branch 6 times, most recently from c7d1484 to 418d802 Compare March 3, 2026 21:59
nikooo777 and others added 30 commits March 12, 2026 16:19
- Cap await_prometheus_started to ~30s to prevent unbounded mailbox growth
- Messages stay in mailbox during wait instead of being consumed and lost
- Wrap declare in try-catch, redeclare lazily on inc failure via ensure_event_counter
- Keep metric writes off the HTTP reply path to isolate request tail latency
- Retain try-catch inside the spawn to guard against metric-path hiccups
fix: Harden metrics and event subsystem under stress
- Add collect_ids clause for as-tuples so IDs inside manifest-cast
  messages are not silently skipped
- Add test for manifest+blacklist with directly resolvable items
- Add test for manifest+blacklist with legacy manifest content-type
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants