[SPARK-55742][SQL] Support TransformingEncoder inside RowEncoder

[eejbyfeldt]

What changes were proposed in this pull request?

Change expected type in ValidateExternalType for TransformingEncoder. Specifically I am trying to create a custom encoders for named tuples in Scala3 https://docs.scala-lang.org/scala3/reference/other-new-features/named-tuples.html this means we want the fields to be named, but the corresponding jvm accessor does not exists. This means that existing ProductEncoder can not be used.

Pre Spark 4 this could easily be solved by a custom Expression based encoder. This fix allows the same in Spark 4 by make it possible to use TransformingEncoders inside RowEncoder. This means that custom named tuple encoder can be created using a transforming encoder to a Row and then appropriate custom encoders for each field.

Why are the changes needed?

This allows more flexible use of RowEncoder when creating custom encoders.

Does this PR introduce any user-facing change?

Yes, it will allow more custom encoders to be created.

How was this patch tested?

Existing and new unit tests.

Was this patch authored or co-authored using generative AI tooling?

No.

[eejbyfeldt]

@hvanhovell @chris-twiner Here is the third and final (at least for now) trying to migrate to use only agnostic encoders. Would be awesome if someone could have a look :)