Add tests for functionality typically provided by webserver frameworks

[Kaliumhexacyanoferrat]

Currently the results reflect the highly-optimized happy-path parts of the targeted webserver frameworks. Real-world applications use additional functionality that is typically provided by webserver frameworks, so those should be tested as well. Examples:

• Compression
• Cache directives
• Template rendering
• Auth
• Entity handling (if built-in, e.g. Ruby on Rails)
• ...

As using such features will certainly impact performance of an application, depending on the underlying implementation of the webserver framework, HttpArena should provide tests for this functionality as well.

Caution

A webserver framework MUST only provide a test implementation if it has support for the scenario already baked in. Our goal is to test frameworks, not third-party libraries such as templating engines or ORM tools (or custom, hand-made implementations just to beat the benchmark).

Proposal

Add additional test cases ("scenarios") that

• Test performance when it comes to a specific concern
• Are probably not part of the composite score
• Are probably available through a new menu tab or something to not distract from the main leaderboard
• Must use the documented way of how a concern is to be implemented in the framework

The following shows an example spec for eTag handling:

Endpoint

Webserver frameworks must provide the following endpoint to opt-in to the scenario test:

GET /scenarios/caching/etag HTTP/1.1
Host ...

HTTP/1.1 200 OK
Date: ...
Server: ...
ETag: W/"123456789"
Content-Length: 2

OK

Validation

Endpoint must:

• Return a fixed, non-computed string ("OK")
• Return a ETag header with the generated response
• The generated ETag header must not change until the server is restarted
• If the request contains header If-None-Matched: "ETag" the server returns
  • HTTP 200 OK with an ETag if the given ETag does not match
  • HTTP 304 Not Modified if the ETag matched

Tests

Troughput without ETag

Goal is to benchmark overhead of ETag generation. Load generator just sends requests without a If-None-Matched header.

Troughput with ETag

Load generator fetches the ETag for the target endpoint once and then sends requests inclduing the If-None-Matched header with the corresponding ETag.