Conversation
trailblaze-host/src/main/java/xyz/block/trailblaze/host/revyl/RevylCliClient.kt
Outdated
Show resolved
Hide resolved
Co-authored-by: Cursor <cursoragent@cursor.com> Signed-off-by: Anam Hira <hira.anam49@gmail.com>
The HTTP-based RevylWorkerClient had wrong API paths, direct worker
access bypassing the backend proxy, and missing auth headers. Four
tool handlers were unimplemented ("not supported — skipping").
RevylCliClient delegates all device actions to the `revyl` CLI binary
via ProcessBuilder. The CLI is auto-downloaded from GitHub Releases if
not already on PATH — the only prerequisite is REVYL_API_KEY.
Changes:
- RevylCliClient.kt: CLI subprocess wrapper with auto-install
- RevylTrailblazeAgent.kt: all 12 tools implemented via CLI
- RevylMcpServerFactory.kt: provisions device via CLI
- RevylMcpBridge.kt: updated constructor and imports
- RevylDeviceService.kt: delegates to CLI client
- RevylScreenState.kt: reads screenshot from temp file
- docs/revyl-integration.md: updated for CLI approach
- RevylDemo.kt: moved to src/test as usage example
Made-with: Cursor
Signed-off-by: Anam Hira <hira.anam49@gmail.com>
… multi-session - RevylMcpBridge: implement runYaml() to parse trail YAML and replay tools; add blazeExecute() for V3 AI exploration via BlazeGoalPlanner - RevylCliClient: refactor from single session to multi-session map with activeSessionIndex; pass -s flag on all device commands; add setNetworkConnected() for airplane mode toggle - RevylTrailblazeAgent: wire network toggle to CLI client - RevylDeviceService: support listing/stopping multiple sessions - RevylMcpServerFactory: add multi-platform create() overload - Fix selectDevice to fail fast on missing session instead of silently continuing on wrong device Made-with: Cursor Signed-off-by: Anam Hira <hira.anam49@gmail.com>
BlazeGoalPlanner requires host-level LLM configuration (ScreenAnalyzer, TrailblazeToolRepo, TrailblazeElementComparator) that isn't available in the CLI-backed bridge. Replace the broken direct instantiation with a clear stub that documents the wiring prerequisite. Trail mode (YAML replay) and all device operations remain fully functional. Made-with: Cursor Signed-off-by: Anam Hira <hira.anam49@gmail.com>
RevylBlazeSupport.createBlazeRunner() takes the host's LLM dependencies (ScreenAnalyzer, TrailblazeToolRepo, TrailblazeElementComparator) and returns a BlazeGoalPlanner backed by Revyl cloud devices. One factory call to swap the device layer from Maestro to Revyl. Also reverts network toggle to honest 'not yet implemented' stub -- the CLI plumbing exists but the cloud device endpoint isn't deployed. Made-with: Cursor Signed-off-by: Anam Hira <hira.anam49@gmail.com>
…ntegration - Add trailblaze-revyl/ module with Revyl-specific tool classes (RevylNativeTapTool, TypeTool, SwipeTool, etc.) that use natural language targeting and return resolved x,y coordinates from AI grounding - RevylCliClient now parses --json output from all action commands, returning RevylActionResult with coordinates, latency, and success status - RevylSession includes screen_width/screen_height from device provisioning - RevylScreenState uses session dimensions when available (falls back to PNG parsing) - RevylMcpBridge forwards coordinate data in tool result messages - RevylTrailblazeAgent includes coordinates in TrailblazeToolResult.Success messages - Add RevylDevicePreset enum (ANDROID_PHONE, IOS_IPHONE) for named device presets - Replace manual binary download with official install.sh installer script - Update RevylDemo.kt to print resolved coordinates from each action Made-with: Cursor Signed-off-by: Anam Hira <hira.anam49@gmail.com>
- Add REVYL_ANDROID and REVYL_IOS to TrailblazeDriverType enum - Register both in OpenSourceTrailblazeDesktopAppConfig initialDriverTypes - Add device discovery in TrailblazeDeviceManager: cloud devices appear when REVYL_API_KEY is set (no local ADB/simulator needed) - Add Revyl types to targetDeviceFilter always-available set - Add getCurrentScreenState when branch for Revyl (uses RevylScreenState) - Fix trailblaze-revyl build.gradle.kts missing trailblaze-host dependency Made-with: Cursor Signed-off-by: Anam Hira <hira.anam49@gmail.com>
…/validation
- Fix persisted device IDs without OS version crashing startSession
- Add require() guard ensuring deviceModel and osVersion are both present or both absent
- Handle trailing :: delimiter edge case with takeIf { isNotBlank() }
- Remove unsupported --device-name CLI flag from startSession
- Add RevylLiveStepResult data class for instruction/validation JSON responses
- Add instruction() and validation() methods to RevylCliClient
- Wire DirectionStep to instruction, VerificationStep to validation (opt-in via useRevylNativeSteps flag, defaults false)
- Add useRevylNativeSteps config flag to TrailblazeConfig
- Add RevylToolAgent with unit tests
- Log parse failures in RevylActionResult.fromJson
- Update docs/revyl-integration.md with native steps documentation
Made-with: Cursor
Signed-off-by: Anam Hira <hira.anam49@gmail.com>
Signed-off-by: Anam Hira <hira.anam49@gmail.com> Made-with: Cursor
- Add REVYL_API_KEY to customEnvVarNames so it appears in Settings > Environment Variables with the same masked value / Configure button UX as LLM provider keys - Check REVYL_API_KEY before session provisioning and surface clear error message instead of raw CLI exception - Wrap RevylCliClient construction and startSession in try-catch so provisioning failures return null with progress message instead of propagating Signed-off-by: Anam Hira <hira.anam49@gmail.com> Made-with: Cursor
When multiple RevylCliClient instances run concurrently (e.g. Android + iOS in parallel), each instance only tracks its own session so sessions.size == 1. The old guard skipped the -s flag in that case, causing the CLI to use its global active session — which could be the other platform's device. Always passing -s when activeSessionIndex >= 0 ensures each client targets its own session regardless of global state. Signed-off-by: Anam Hira <hira.anam49@gmail.com> Made-with: Cursor
Pass the Revyl cloud device viewer URL through TrailblazeDeviceInfo metadata so it persists in session logs. Render it as a clickable "Open Revyl Viewer" link in the SessionDetailHeader when present, allowing users to jump directly to the live device screen. Signed-off-by: Anam Hira <hira.anam49@gmail.com> Made-with: Cursor
handstandsam
left a comment
There was a problem hiding this comment.
Added lots and lots and lots of comments. Hopefully we can clean it up to get it going.
| val trailblazeLlmProvider = trailblazeLlmModelList.provider | ||
| JvmLLMProvidersUtil.getEnvironmentVariableKeyForLlmProvider(trailblazeLlmProvider) | ||
| } | ||
| } + "REVYL_API_KEY" |
There was a problem hiding this comment.
Let's make this as a constant in a follow up.
| */ | ||
| fun createBlazeRunner( | ||
| cliClient: RevylCliClient, | ||
| platform: String, |
There was a problem hiding this comment.
Can we make this a constant or an enum? We already have TrailblazeDevicePlatform I think.
| elementComparator: TrailblazeElementComparator, | ||
| config: BlazeConfig = BlazeConfig.DEFAULT, | ||
| ): BlazeGoalPlanner { | ||
| val devicePlatform = if (platform == "ios") TrailblazeDevicePlatform.IOS else TrailblazeDevicePlatform.ANDROID |
There was a problem hiding this comment.
Ahh yeah, then we wouldn't need this here. Maybe just a util method to do the transformation before calling this?
| val devicePlatform = if (platform == "ios") TrailblazeDevicePlatform.IOS else TrailblazeDevicePlatform.ANDROID | ||
| val deviceInfo = TrailblazeDeviceInfo( | ||
| trailblazeDeviceId = TrailblazeDeviceId(instanceId = "revyl-blaze", trailblazeDevicePlatform = devicePlatform), | ||
| trailblazeDriverType = if (platform == "ios") TrailblazeDriverType.REVYL_IOS else TrailblazeDriverType.REVYL_ANDROID, |
There was a problem hiding this comment.
Same, would be easier to do a typesafe if/else or when.
| widthPixels = 0, | ||
| heightPixels = 0, |
There was a problem hiding this comment.
This is fine, but I'm wondering if we should just have a reasonable default for this. For an iOS or an Android phone. Also, could this be an iPad or a tablet? What device are we guaranteed to get? I think right now it's just iPhone and Android phone if I understand correctly.
| val target: String, | ||
| override val reasoning: String? = null, |
There was a problem hiding this comment.
This is where you could add val longPress: Boolean = false if you wanted as an option.
| object RevylNativeToolSet { | ||
|
|
||
| /** Core tools for mobile interaction -- tap, type, swipe, navigate, etc. */ | ||
| val CoreToolSet = |
There was a problem hiding this comment.
Could we call it RevylCoreToolSet? Sorry, there's just so many things in this project and having the prefix is super helpful.
| trailblazeLogger = TrailblazeLogger.createNoOp(), | ||
| trailblazeDeviceInfoProvider = { deviceInfo }, | ||
| sessionProvider = { | ||
| TrailblazeSession(sessionId = SessionId("revyl-mcp"), startTime = kotlinx.datetime.Clock.System.now()) |
There was a problem hiding this comment.
TBH, I'm not sure how the MCP and normal sessions are different here. Not sure how this MCP Server part works.
README.md
Outdated
| ### Cloud Device Support (Revyl) | ||
|
|
||
| You can run Trailblaze against [Revyl](https://revyl.ai) cloud devices instead of local ADB or Maestro. Use | ||
| `RevylMcpServerFactory` to create an MCP server that provisions a device and maps Trailblaze tools to Revyl HTTP APIs. | ||
| See the [Revyl integration guide](docs/revyl-integration.md) for prerequisites, architecture, and usage. | ||
|
|
There was a problem hiding this comment.
Ideally, this would go in supported devices or something like that and not on the main readme.
| val revylViewerUrl = sessionDetail.session.trailblazeDeviceInfo | ||
| ?.metadata?.get("revyl_viewer_url") | ||
| ?.takeIf { it.isNotBlank() } | ||
| if (revylViewerUrl != null) { | ||
| val uriHandler = LocalUriHandler.current | ||
| TextButton( | ||
| onClick = { uriHandler.openUri(revylViewerUrl) }, | ||
| contentPadding = ButtonDefaults.TextButtonContentPadding, | ||
| ) { | ||
| Text( | ||
| text = "Open Revyl Viewer", | ||
| style = MaterialTheme.typography.labelSmall, | ||
| color = MaterialTheme.colorScheme.primary, | ||
| ) | ||
| } | ||
| } |
There was a problem hiding this comment.
Interesting. I wonder if there's a way we can plug this in without having to have this specific logic in here. This is different than the way we handle all the others. I see the value here if somebody is using a Rebel device type, but this is just for our session details everywhere. We should probably make a plugin like system where Rebel can be added.
Merge blockers: - Remove auto-install/auto-update of revyl CLI; replace with verifyCliAvailable() that shows install instructions on error and checkForUpgrade() that logs a non-blocking upgrade warning - Replace hardcoded Revyl viewer link in SessionDetailHeader with generic metadata-driven link system (any *_url key renders a link) - Remove Revyl section from root README.md (docs/revyl-integration.md already covers it) Code quality (per review comments): - RevylBlazeSupport: accept TrailblazeDevicePlatform enum instead of raw String; add default screen dimensions per platform - Rename getActiveSession() -> getActiveRevylSession() - RevylExecutableTool: convert from interface to abstract class, implement ReasoningTrailblazeTool, rename client -> revylClient - Merge RevylNativeLongPressTool into RevylNativeTapTool (longPress param) - Remove RevylNativeScreenshotTool from LLM tool set - Rename tool sets: CoreToolSet -> RevylCoreToolSet, etc. - Fix tool naming: revyl_press_key -> revyl_pressKey - Add ANDROID_BACK key support, iOS back clarification - Add swipe direction validation, assert examples for LLM - Use TrailblazeJsonInstance instead of private Json instances - Use File.createTempFile() for screenshot paths - Add ACTIVE_SESSION constant, REVYL_API_KEY_ENV constant - Extract platform-to-driver-type utils in RevylDeviceService - Fix build warnings: suppress deprecated LongPressOnElementWithText, remove redundant else branches in when expressions Made-with: Cursor Signed-off-by: Anam Hira <hira.anam49@gmail.com>
The Revyl CLI now requires --target or --x/--y for type and clear-text commands. InputTextTrailblazeTool and EraseTextTrailblazeTool don't carry a target field (the LLM focuses the field first via tap), so pass "focused input field" as a default target for AI grounding. Made-with: Cursor Signed-off-by: Anam Hira <hira.anam49@gmail.com>
| val trailblazeLlmProvider = trailblazeLlmModelList.provider | ||
| JvmLLMProvidersUtil.getEnvironmentVariableKeyForLlmProvider(trailblazeLlmProvider) | ||
| } | ||
| } + xyz.block.trailblaze.host.revyl.RevylCliClient.REVYL_API_KEY_ENV |
There was a problem hiding this comment.
NIT: Fully qualified import is overkill.
| TrailblazeDevicePlatform.IOS -> Pair(1170, 2532) | ||
| else -> Pair(1080, 2400) |
There was a problem hiding this comment.
Nit: this is as constants and document what they are.
| val results = mutableListOf<RevylDeviceTarget>() | ||
| val seenModels = mutableSetOf<String>() | ||
|
|
||
| for (platform in listOf("android", "ios")) { |
There was a problem hiding this comment.
Could be constants for the "REVYL_PLATFORM_IOS"
handstandsam
left a comment
There was a problem hiding this comment.
Currently, it looks like you're mapping the Trailblaze Standard tools over to Revyl. I would highly suggest using your tools directly.
| // Render external links from device metadata (convention: *_url keys become clickable links) | ||
| val metadata = sessionDetail.session.trailblazeDeviceInfo?.metadata.orEmpty() | ||
| val externalLinks = metadata.entries | ||
| .filter { it.key.endsWith("_url") && it.value.isNotBlank() } | ||
| .map { (key, url) -> | ||
| val prefix = key.removeSuffix("_url") | ||
| val label = metadata["${prefix}_label"] | ||
| ?: "Open ${prefix.replace('_', ' ').replaceFirstChar { c -> c.uppercase() }}" | ||
| label to url | ||
| } | ||
| if (externalLinks.isNotEmpty()) { | ||
| val uriHandler = LocalUriHandler.current | ||
| for ((label, url) in externalLinks) { | ||
| TextButton( | ||
| onClick = { uriHandler.openUri(url) }, | ||
| contentPadding = ButtonDefaults.TextButtonContentPadding, | ||
| ) { | ||
| Text( | ||
| text = label, | ||
| style = MaterialTheme.typography.labelSmall, | ||
| color = MaterialTheme.colorScheme.primary, | ||
| ) | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
This is still a lot of key lookups on the map, could we create a utility that does this to take the "deviceInfo" and come back with the "external links" data?
| data class RevylSession( | ||
| val index: Int, | ||
| val sessionId: String, | ||
| val workflowRunId: String, | ||
| val workerBaseUrl: String, | ||
| val viewerUrl: String, | ||
| val platform: String, | ||
| val screenWidth: Int = 0, | ||
| val screenHeight: Int = 0, | ||
| ) |
There was a problem hiding this comment.
Can this move to :trailblaze-revyl
| data class RevylActionResult( | ||
| val action: String = "", | ||
| val x: Int = 0, | ||
| val y: Int = 0, | ||
| val target: String? = null, | ||
| val success: Boolean = true, | ||
| @SerialName("latency_ms") | ||
| val latencyMs: JsonElement? = null, | ||
| @SerialName("duration_ms") | ||
| val durationMs: Int = 0, | ||
| val text: String? = null, | ||
| val direction: String? = null, | ||
| ) { |
There was a problem hiding this comment.
Can this move to :trailblaze-revyl
| class RevylDeviceService( | ||
| private val cliClient: RevylCliClient, | ||
| ) { |
There was a problem hiding this comment.
Can this move to :trailblaze-revyl
There was a problem hiding this comment.
Can this be deleted?
| @Serializable | ||
| data class RevylLiveStepResult( | ||
| val success: Boolean = false, | ||
| @SerialName("step_type") val stepType: String = "", | ||
| @SerialName("step_id") val stepId: String = "", | ||
| @SerialName("step_output") val stepOutput: JsonObject? = null, | ||
| ) { |
There was a problem hiding this comment.
Can this move to :trailblaze-revyl
| * The CLI binary must be on PATH (or set via REVYL_BINARY env var) | ||
| * and authenticated via REVYL_API_KEY or `revyl auth login`. | ||
| */ | ||
| object RevylMcpServerFactory { |
There was a problem hiding this comment.
I think you can delete this.
| * @property agent The Trailblaze agent that dispatches tools via CLI. | ||
| * @property platform Device platform ("ios" or "android"). | ||
| */ | ||
| class RevylMcpBridge( |
There was a problem hiding this comment.
I think you can delete this.
| object RevylNativeToolSet { | ||
|
|
||
| /** Core tools for mobile interaction -- tap, type, swipe, navigate, etc. */ | ||
| val RevylCoreToolSet = | ||
| DynamicTrailblazeToolSet( | ||
| name = "Revyl Native Core", | ||
| toolClasses = | ||
| setOf( | ||
| RevylNativeTapTool::class, | ||
| RevylNativeTypeTool::class, | ||
| RevylNativeSwipeTool::class, | ||
| RevylNativeNavigateTool::class, | ||
| RevylNativeBackTool::class, | ||
| RevylNativePressKeyTool::class, | ||
| ObjectiveStatusTrailblazeTool::class, | ||
| ), | ||
| ) | ||
|
|
||
| /** Revyl assertion tools for visual verification. */ | ||
| val RevylAssertionToolSet = | ||
| DynamicTrailblazeToolSet( | ||
| name = "Revyl Native Assertions", | ||
| toolClasses = | ||
| setOf( | ||
| RevylNativeAssertTool::class, | ||
| ), | ||
| ) | ||
|
|
||
| /** Full LLM tool set -- core tools plus assertions and memory tools. */ | ||
| val RevylLlmToolSet = | ||
| DynamicTrailblazeToolSet( | ||
| name = "Revyl Native LLM", | ||
| toolClasses = | ||
| RevylCoreToolSet.toolClasses + | ||
| RevylAssertionToolSet.toolClasses + | ||
| xyz.block.trailblaze.toolcalls.TrailblazeToolSet.RememberTrailblazeToolSet.toolClasses, | ||
| ) | ||
| } |
There was a problem hiding this comment.
These are unused, which is why you don't have your tools. We should use it somewhere
There was a problem hiding this comment.
Use this in @trailblaze-host/src/main/java/xyz/block/trailblaze/host/TrailblazeHostYamlRunner.kt and other places that use the "TrailblazeDriverType.PLAYWRIGHT_NATIVE", you'll want to follow the similar pattern in the when statements.
You will need a method inside that host runner specifically like playwrightNative
| widthPixels = 0, | ||
| heightPixels = 0, |
There was a problem hiding this comment.
I think you extracted a constant for this value for iOS and for Android. Could you use it here?
- Fix --clear-first flag to always pass explicit value (CLI defaults to true) - Add appId and buildVersionId params to startSession() for Revyl build mgmt - Add revyl_doubleTap native tool for zoom/double-tap UI patterns - Move Revyl data classes from trailblaze-host to trailblaze-revyl module - Add RevylDefaults for platform-specific screen dimension fallbacks - Wire RevylNativeToolSet for all Revyl sessions (MCP, CLI, YAML runner) - InputTextTrailblazeTool fallback passes "text input field" target - Add ExternalLinkUtils for viewer URL handling in session UI Made-with: Cursor Signed-off-by: Anam Hira <hira.anam49@gmail.com> Made-with: Cursor
Integration with Revyl CLI for IOS and Android Device control
https://github.com/RevylAI/revyl-cli
2s Device Start time
Full runtime capability
unlimited concurrency
Link to repro // Demo: https://cap.so/s/kk74h11fdb5wh59