Skip to content

update v1.59.0#55

Open
Dog-Du wants to merge 685 commits intotikv:rs-releasefrom
Dog-Du:merge-v1.59.0
Open

update v1.59.0#55
Dog-Du wants to merge 685 commits intotikv:rs-releasefrom
Dog-Du:merge-v1.59.0

Conversation

@Dog-Du
Copy link
Copy Markdown

@Dog-Du Dog-Du commented Jul 29, 2025

Change

  1. merge
git merge --no-commit "v1.59.0" 
git checkout --theirs .
  1. remove
    remove DisableOutlierDetectionAttribute, GRPC_ARG_MIN_MESSAGE_SIZE_TO_COMPRESS, GRPC_ARG_GZIP_COMPRESSION_LEVEL.
    they are useless in v1.59.0
  2. add
    grpc/tools/run_tests/performance/scenario_config.py should not delete rustLanguage.
    do not remove it.
  3. compatibility
  • a lot of undefined symbols, like :
          Undefined symbols for architecture arm64:
            "_utf8_range2", referenced from:
                __upb_Decoder_VerifyUtf8Inline in libgrpcio_sys-090a894cc81a7d0a.rlib[280](decode.c.o)
          ld: symbol(s) not found for architecture arm64
          clang-16: error: linker command failed with exit code 1 (use -v to see invocation)
error: could not compile `benchmark` (bin "qps_worker") due to 1 previous error

the upb library was split into multiple parts in v1.59.0.
merge them into one library to stay compatible

  • link re2 for grpc_unsecure.
    need re2 library when cargo xtask / cargo build -p grpcio-proto / cargo build -p interop.
    re2 currently is not linked to grpc_unsecure in v1.59.0.
    add re2 for grpc_unsecure.

markdroth and others added 30 commits August 18, 2023 08:48
CNR the flake, but I've changed the test (which is very old) to use some
of our more modern helper functions that have saner timeouts.

Also re-add a `return` statement that was accidentally removed in
grpc#33753, which I noticed while working on this. Its absence doesn't cause
a real problem, but it does cause us to needlessly trigger a duplicate
connection attempt or report a duplicate CONNECTING update in some
cases.
…pc#34101)

Currently the bazel invocations provide a link back to the original
kokoro jobs for resultstore and sponge UIs.
This adds another back link to Fusion UI, which has the advantage of 
- being able to navigate to the kokoro job overview
- there is a button to trigger a new build (in case the job needs to be
re-run).


![image](https://github.com/grpc/grpc/assets/9939684/42f0bbc2-14f5-45db-89f7-73e48e32e7c9)
…aster (grpc#34102)

Basically run each of the subtests (buildtest, distribtest_cpp,
distribtest_python) as a separate bazel target.

- currently the bazel distribtest are the slowest targets in
grpc_bazel_rbe_nonbazel
- the shards are basically independent tests anyway
- when split into multiple targets, they each get a separate target log
so it's easier debug issues since there isn't multiple bazel invocations
in each log.
…the WorkSerializer (grpc#34077)

- add debug-only `WorkSerializer::IsRunningInWorkSerializer()` method
and use it in client_channel to verify that subchannels are destroyed in
the `WorkSerializer`
  - note: this mechanism uses `std::thread::id`, so I had to exclude
    work_serializer.cc from the core_banned_constructs check
- fix `WorkSerializer::Run()` to unref the callback before releasing
ownership of the `WorkSerializer`, so that any refs captured by the
`std::function<>` will be released before releasing ownership
- fix the WRR timer callback to hop into the `WorkSerializer` to release
its ref to the picker, since that transitively releases refs to
subchannels
- fix subchannel connectivity state notifications to unref the watcher
inside the `WorkSerializer`, since the watcher often transitively holds
refs to subchannels
Update from gtcooke94:
This PR adds support to build gRPC and it's tests with OpenSSL3. There were some
hiccups with tests as the tests with openssl haven't been built or exercised in a
few months, so they needed some work to fix.

Right now I expect all test files to pass except the following:
- h2_ssl_cert_test
- ssl_transport_security_utils_test

I confirmed locally that these tests fail with OpenSSL 1.1.1 as well,
thus we are at least not introducing regressions. Thus, I've added compiler directives around these tests so they only build when using BoringSSL.

---------

Co-authored-by: Gregory Cooke <gregorycooke@google.com>
Co-authored-by: Esun Kim <veblush@google.com>
…34109)

Now that iOS DNS resolver is ready, we can also enable it with
`GRPC_IOS_EVENT_ENGINE_CLIENT`, so we can ask users to experiment with
it.
… on Windows (grpc#34107)

Local Bazel invocation succeeds:

```
C:\Users\yijiem\projects\grpc>bazel --output_base=C:\bazel2 test --dynamic_mode=off --verbose_failures //test/cpp/naming:resolver_component_tests_runner_invoker@poller=epoll1
INFO: Analyzed target //test/cpp/naming:resolver_component_tests_runner_invoker@poller=epoll1 (0 packages loaded, 0 targets configured).
INFO: Found 1 test target...
Target //test/cpp/naming:resolver_component_tests_runner_invoker@poller=epoll1 up-to-date:
  bazel-bin/test/cpp/naming/resolver_component_tests_runner_invoker@poller=epoll1.exe
INFO: Elapsed time: 199.262s, Critical Path: 193.48s
INFO: 2 processes: 1 internal, 1 local.
INFO: Build completed successfully, 2 total actions
//test/cpp/naming:resolver_component_tests_runner_invoker@poller=epoll1  PASSED in 193.4s

Executed 1 out of 1 test: 1 test passes.
```

The local invocation of RBE failed with linker error `LINK : error
LNK2001: unresolved external symbol mainCRTStartup`, but that does not
limited to this target:
https://gist.github.com/yijiem/2c6cbd9a31209a6de8fd711afbf2b479.

<!--

If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the
appropriate
lang label.

-->
…34097)

This is to make sure upgrading packaging module won't break our logic on
version-based version skipping.
This also fixes a small issue with `dev-` prefix - it should only be
allowed on the left side of the comparison.

Context: packaging module needs to be upgraded to be compatible with
`blackd`.
…o pull in __STDC_FORMAT_MACROS define) (grpc#32159)

Some versions of MacOS (as well as some `glibc`-based platforms) require
`__STDC_FORMAT_MACROS` to be defined prior to including `<cinttypes>` or
`<inttypes.h>`, otherwise build fails with undeclared `PRIdMAX`,
`PRIdPTR` etc.

See note here: https://en.cppreference.com/w/cpp/types/integer

label: (release notes: no)
Add example for TLS.

<!--

If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the
appropriate
lang label.

-->
…rpc#34127)

The tests are skipped incorrectly because `config.server_lang` is
incorrectly compared with the string value "java", instead of
`skips.Lang.JAVA`.

This has been broken since grpc#26998. 

```
xds_url_map_testcase.py:372] ----- Testing TestTimeoutInRouteRule -----
xds_url_map_testcase.py:373] Logs timezone: UTC
skips.py:121] Skipping TestConfig(client_lang='java', server_lang='java', version='v1.57.x')
[  SKIPPED ] setUpClass (timeout_test.TestTimeoutInRouteRule)
xds_url_map_testcase.py:372] ----- Testing TestTimeoutInApplication -----
xds_url_map_testcase.py:373] Logs timezone: UTC
skips.py:121] Skipping TestConfig(client_lang='java', server_lang='java', version='v1.57.x')
[  SKIPPED ] setUpClass (timeout_test.TestTimeoutInApplication)
```
- Add Github Action to conditionally run PSM Interop unit tests:
- Only run when changes are detected in
`tools/run_tests/xds_k8s_test_driver` or any of the proto files used by
the driver
  - Only run against PRs and pushes to `master`, `v1.*.*` branches
  - Runs using `python3.9` and `python3.10`
  - Ready to be added to the list of required GitHub checks
- Add `tools/run_tests/xds_k8s_test_driver/tests/unit/__main__.py` test
loader that recursively discovers all unit tests in
`tools/run_tests/xds_k8s_test_driver/tests/unit`
- Add basic coverage for `XdsTestClient` and `XdsTestServer` to verify
the test loader picks up all folders

Related:
- First unit tests without automated CI added in grpc#34097
We enabled OpenSSL3 testing with grpc#31256 and missed a failing test

It wasn't running before, so this isn't a regression - disabling it so
master doesn't fail while we figure out how to fix it.
…4140)

This WorkStealingThreadPool change improves the (lagging)
`cpp_protobuf_async_streaming_qps_unconstrained_secure` 32-core
benchmark.

Baseline OriginalThreadPool QPS: 830k
Previous average WorkStealingThreadPool QPS: 755k
New WorkStealingThreadPool average (2 runs) QPS: 850k
…rpc#34130)

Fixes an issue when an active context selected automatically picked up
as context for `secondary_k8s_api_manager`.

This was introducing an error in GAMMA Baseline PoC

```
sys:1: ResourceWarning: unclosed <ssl.SSLSocket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('100.71.2.143', 56723), raddr=('35.199.174.232', 443)>
```

Here's how the secondary context is incorrectly falls back to the
default context when `--secondary_kube_context` is not set:
```
k8s.py:142] Using kubernetes context "gke_grpc-testing_us-central1-a_psm-interop-security", active host: https://35.202.85.90
k8s.py:142] Using kubernetes context "None", active host: https://35.202.85.90
```
Change was created by the release automation script. See
go/grpc-release.
The `work_stealing` experiment on its own is not very valuable, so let's
delete it and save CI resources. We have a benchmark for
`GRPC_EXPERIMENTS=event_engine_listener,work_stealing`, which is really
what we care about right now.
Previously black wouldn't install, as it required newer `packaging`
package.

This fixes `pip install -r requirements-dev.txt`. In addition, `black`
in dev dependencies file is changed to `black[d]`, which bundles
`blackd` binary (["black as a
server"](https://black.readthedocs.io/en/stable/usage_and_configuration/black_as_a_server.html)).
This helps developers run benchmark loadtests locally. See comments in
scenario_runner.py for usage.

---------

Co-authored-by: drfloob <drfloob@users.noreply.github.com>
grpc#34150)

Reminder: changes to this file are picked up immediately by all
tests/all branches/all langs.
…c#34158)

This reverts commit fe1ba18.

Reason: break import



<!--

If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the
appropriate
lang label.

-->
XuanWang-Amos and others added 25 commits September 23, 2023 13:36
Passed manual run:
https://fusion2.corp.google.com/invocations/0e337438-8573-4b54-ba1c-242b1ad58586
<!--

If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the
appropriate
lang label.

-->
Joint effort with @yashykt and @sanjaypujare

---------

Co-authored-by: Yash Tibrewal <yashkt@google.com>
Co-authored-by: Sanjay Pujare <sanjaypujare@users.noreply.github.com>
Expand our fuzzing capabilities by allowing fuzzers to choose the bits
that go into random number distribution generators.

---------

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
Cache realignment saves 64 bytes per transport, 40 bytes per stream, 8
bytes per tcp endpoint, and 24 bytes per transport op.
We disabled this a little while ago for lack of CI bandwidth, but grpc#34404
ought to have freed up enough capacity that we can keep running this.

---------

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
Fix a crash on older iOS versions due to problematic thread-local
variable initialization.

See firebase/firebase-ios-sdk#11509

Basically, there appears to be a bug in Xcode where it generates
assembly for thread-local variable initialization that is susceptible to
a crash. For example, on arm64 the generated assembly relies on
registers like x8 and x10 being preserved by the thread-local variable
initialization routine; however, in some cases this thread-local
variable initialization calls functions like
`ImageLoaderMachOCompressed::doBindFastLazySymbol` which clobber these
registers, leaving their values indeterminate when the caller resumes.
When those indeterminate values are later used as memory addresses they
are invalid and result in a crash.

This PR works around this bug by removing the `ScopedTimeCache` member
variable from the `ExecCtx` class on iOS. This is a reasonable
workaround because `ScopedTimeCache` is only a slight optimization for
data centers that entirely doesn't matter for mobile.

See https://github.com/dconeybe/TlsCrashIos12 for a demo of this crash.

Googlers see b/300501963 for full details.
Was never needed in the first place.
Backports grpc#34477 to v1.59.x

This is test-only, so going forward with 1.59.0-pre despite the one
failing distribtest image.

Co-authored-by: Eugene Ostroukhov <eostroukhov@google.com>
Signed-off-by: DogDu <du.wen.hao@outlook.com>
Signed-off-by: DogDu <du.wen.hao@outlook.com>
Signed-off-by: DogDu <du.wen.hao@outlook.com>
@ti-chi-bot
Copy link
Copy Markdown

ti-chi-bot bot commented Jul 29, 2025

Welcome @Dog-Du! It looks like this is your first PR to tikv/grpc 🎉

@ti-chi-bot
Copy link
Copy Markdown

ti-chi-bot bot commented Jul 29, 2025

Thanks for your pull request. Before we can look at it, you'll need to add a 'DCO signoff' to your commits.

📝 Please follow instructions in the contributing guide to update your commits with the DCO

Full details of the Developer Certificate of Origin can be found at developercertificate.org.

The list of commits missing DCO signoff:

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@Dog-Du Dog-Du closed this Aug 15, 2025
@Dog-Du Dog-Du reopened this Aug 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.