Skip to content

Revyl CLI integration#110

Open
anamhira47 wants to merge 15 commits intoblock:mainfrom
anamhira47:main
Open

Revyl CLI integration#110
anamhira47 wants to merge 15 commits intoblock:mainfrom
anamhira47:main

Conversation

@anamhira47
Copy link

@anamhira47 anamhira47 commented Mar 19, 2026

Integration with Revyl CLI for IOS and Android Device control

https://github.com/RevylAI/revyl-cli

2s Device Start time

Full runtime capability

unlimited concurrency

Link to repro // Demo: https://cap.so/s/kk74h11fdb5wh59

anamhira47 and others added 12 commits March 25, 2026 09:33
Co-authored-by: Cursor <cursoragent@cursor.com>
Signed-off-by: Anam Hira <hira.anam49@gmail.com>
The HTTP-based RevylWorkerClient had wrong API paths, direct worker
access bypassing the backend proxy, and missing auth headers. Four
tool handlers were unimplemented ("not supported — skipping").

RevylCliClient delegates all device actions to the `revyl` CLI binary
via ProcessBuilder. The CLI is auto-downloaded from GitHub Releases if
not already on PATH — the only prerequisite is REVYL_API_KEY.

Changes:
- RevylCliClient.kt: CLI subprocess wrapper with auto-install
- RevylTrailblazeAgent.kt: all 12 tools implemented via CLI
- RevylMcpServerFactory.kt: provisions device via CLI
- RevylMcpBridge.kt: updated constructor and imports
- RevylDeviceService.kt: delegates to CLI client
- RevylScreenState.kt: reads screenshot from temp file
- docs/revyl-integration.md: updated for CLI approach
- RevylDemo.kt: moved to src/test as usage example

Made-with: Cursor
Signed-off-by: Anam Hira <hira.anam49@gmail.com>
… multi-session

- RevylMcpBridge: implement runYaml() to parse trail YAML and replay
  tools; add blazeExecute() for V3 AI exploration via BlazeGoalPlanner
- RevylCliClient: refactor from single session to multi-session map
  with activeSessionIndex; pass -s flag on all device commands; add
  setNetworkConnected() for airplane mode toggle
- RevylTrailblazeAgent: wire network toggle to CLI client
- RevylDeviceService: support listing/stopping multiple sessions
- RevylMcpServerFactory: add multi-platform create() overload
- Fix selectDevice to fail fast on missing session instead of
  silently continuing on wrong device

Made-with: Cursor
Signed-off-by: Anam Hira <hira.anam49@gmail.com>
BlazeGoalPlanner requires host-level LLM configuration (ScreenAnalyzer,
TrailblazeToolRepo, TrailblazeElementComparator) that isn't available in
the CLI-backed bridge. Replace the broken direct instantiation with a
clear stub that documents the wiring prerequisite. Trail mode (YAML
replay) and all device operations remain fully functional.

Made-with: Cursor
Signed-off-by: Anam Hira <hira.anam49@gmail.com>
RevylBlazeSupport.createBlazeRunner() takes the host's LLM dependencies
(ScreenAnalyzer, TrailblazeToolRepo, TrailblazeElementComparator) and
returns a BlazeGoalPlanner backed by Revyl cloud devices. One factory
call to swap the device layer from Maestro to Revyl.

Also reverts network toggle to honest 'not yet implemented' stub --
the CLI plumbing exists but the cloud device endpoint isn't deployed.

Made-with: Cursor
Signed-off-by: Anam Hira <hira.anam49@gmail.com>
…ntegration

- Add trailblaze-revyl/ module with Revyl-specific tool classes (RevylNativeTapTool,
  TypeTool, SwipeTool, etc.) that use natural language targeting and return resolved
  x,y coordinates from AI grounding
- RevylCliClient now parses --json output from all action commands, returning
  RevylActionResult with coordinates, latency, and success status
- RevylSession includes screen_width/screen_height from device provisioning
- RevylScreenState uses session dimensions when available (falls back to PNG parsing)
- RevylMcpBridge forwards coordinate data in tool result messages
- RevylTrailblazeAgent includes coordinates in TrailblazeToolResult.Success messages
- Add RevylDevicePreset enum (ANDROID_PHONE, IOS_IPHONE) for named device presets
- Replace manual binary download with official install.sh installer script
- Update RevylDemo.kt to print resolved coordinates from each action

Made-with: Cursor
Signed-off-by: Anam Hira <hira.anam49@gmail.com>
- Add REVYL_ANDROID and REVYL_IOS to TrailblazeDriverType enum
- Register both in OpenSourceTrailblazeDesktopAppConfig initialDriverTypes
- Add device discovery in TrailblazeDeviceManager: cloud devices appear
  when REVYL_API_KEY is set (no local ADB/simulator needed)
- Add Revyl types to targetDeviceFilter always-available set
- Add getCurrentScreenState when branch for Revyl (uses RevylScreenState)
- Fix trailblaze-revyl build.gradle.kts missing trailblaze-host dependency

Made-with: Cursor
Signed-off-by: Anam Hira <hira.anam49@gmail.com>
…/validation

- Fix persisted device IDs without OS version crashing startSession
- Add require() guard ensuring deviceModel and osVersion are both present or both absent
- Handle trailing :: delimiter edge case with takeIf { isNotBlank() }
- Remove unsupported --device-name CLI flag from startSession
- Add RevylLiveStepResult data class for instruction/validation JSON responses
- Add instruction() and validation() methods to RevylCliClient
- Wire DirectionStep to instruction, VerificationStep to validation (opt-in via useRevylNativeSteps flag, defaults false)
- Add useRevylNativeSteps config flag to TrailblazeConfig
- Add RevylToolAgent with unit tests
- Log parse failures in RevylActionResult.fromJson
- Update docs/revyl-integration.md with native steps documentation

Made-with: Cursor
Signed-off-by: Anam Hira <hira.anam49@gmail.com>
Signed-off-by: Anam Hira <hira.anam49@gmail.com>
Made-with: Cursor
- Add REVYL_API_KEY to customEnvVarNames so it appears in Settings > Environment Variables with the same masked value / Configure button UX as LLM provider keys
- Check REVYL_API_KEY before session provisioning and surface clear error message instead of raw CLI exception
- Wrap RevylCliClient construction and startSession in try-catch so provisioning failures return null with progress message instead of propagating

Signed-off-by: Anam Hira <hira.anam49@gmail.com>
Made-with: Cursor
When multiple RevylCliClient instances run concurrently (e.g. Android +
iOS in parallel), each instance only tracks its own session so
sessions.size == 1. The old guard skipped the -s flag in that case,
causing the CLI to use its global active session — which could be the
other platform's device. Always passing -s when activeSessionIndex >= 0
ensures each client targets its own session regardless of global state.

Signed-off-by: Anam Hira <hira.anam49@gmail.com>
Made-with: Cursor
Pass the Revyl cloud device viewer URL through TrailblazeDeviceInfo
metadata so it persists in session logs. Render it as a clickable
"Open Revyl Viewer" link in the SessionDetailHeader when present,
allowing users to jump directly to the live device screen.

Signed-off-by: Anam Hira <hira.anam49@gmail.com>
Made-with: Cursor
Copy link
Collaborator

@handstandsam handstandsam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added lots and lots and lots of comments. Hopefully we can clean it up to get it going.

val trailblazeLlmProvider = trailblazeLlmModelList.provider
JvmLLMProvidersUtil.getEnvironmentVariableKeyForLlmProvider(trailblazeLlmProvider)
}
} + "REVYL_API_KEY"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's make this as a constant in a follow up.

*/
fun createBlazeRunner(
cliClient: RevylCliClient,
platform: String,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we make this a constant or an enum? We already have TrailblazeDevicePlatform I think.

elementComparator: TrailblazeElementComparator,
config: BlazeConfig = BlazeConfig.DEFAULT,
): BlazeGoalPlanner {
val devicePlatform = if (platform == "ios") TrailblazeDevicePlatform.IOS else TrailblazeDevicePlatform.ANDROID
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh yeah, then we wouldn't need this here. Maybe just a util method to do the transformation before calling this?

val devicePlatform = if (platform == "ios") TrailblazeDevicePlatform.IOS else TrailblazeDevicePlatform.ANDROID
val deviceInfo = TrailblazeDeviceInfo(
trailblazeDeviceId = TrailblazeDeviceId(instanceId = "revyl-blaze", trailblazeDevicePlatform = devicePlatform),
trailblazeDriverType = if (platform == "ios") TrailblazeDriverType.REVYL_IOS else TrailblazeDriverType.REVYL_ANDROID,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same, would be easier to do a typesafe if/else or when.

Comment on lines +70 to +71
widthPixels = 0,
heightPixels = 0,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is fine, but I'm wondering if we should just have a reasonable default for this. For an iOS or an Android phone. Also, could this be an iPad or a tablet? What device are we guaranteed to get? I think right now it's just iPhone and Android phone if I understand correctly.

Comment on lines +27 to +28
val target: String,
override val reasoning: String? = null,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is where you could add val longPress: Boolean = false if you wanted as an option.

object RevylNativeToolSet {

/** Core tools for mobile interaction -- tap, type, swipe, navigate, etc. */
val CoreToolSet =
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we call it RevylCoreToolSet? Sorry, there's just so many things in this project and having the prefix is super helpful.

trailblazeLogger = TrailblazeLogger.createNoOp(),
trailblazeDeviceInfoProvider = { deviceInfo },
sessionProvider = {
TrailblazeSession(sessionId = SessionId("revyl-mcp"), startTime = kotlinx.datetime.Clock.System.now())
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TBH, I'm not sure how the MCP and normal sessions are different here. Not sure how this MCP Server part works.

README.md Outdated
Comment on lines +69 to +74
### Cloud Device Support (Revyl)

You can run Trailblaze against [Revyl](https://revyl.ai) cloud devices instead of local ADB or Maestro. Use
`RevylMcpServerFactory` to create an MCP server that provisions a device and maps Trailblaze tools to Revyl HTTP APIs.
See the [Revyl integration guide](docs/revyl-integration.md) for prerequisites, architecture, and usage.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally, this would go in supported devices or something like that and not on the main readme.

Comment on lines +152 to +167
val revylViewerUrl = sessionDetail.session.trailblazeDeviceInfo
?.metadata?.get("revyl_viewer_url")
?.takeIf { it.isNotBlank() }
if (revylViewerUrl != null) {
val uriHandler = LocalUriHandler.current
TextButton(
onClick = { uriHandler.openUri(revylViewerUrl) },
contentPadding = ButtonDefaults.TextButtonContentPadding,
) {
Text(
text = "Open Revyl Viewer",
style = MaterialTheme.typography.labelSmall,
color = MaterialTheme.colorScheme.primary,
)
}
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting. I wonder if there's a way we can plug this in without having to have this specific logic in here. This is different than the way we handle all the others. I see the value here if somebody is using a Rebel device type, but this is just for our session details everywhere. We should probably make a plugin like system where Rebel can be added.

Merge blockers:
- Remove auto-install/auto-update of revyl CLI; replace with
  verifyCliAvailable() that shows install instructions on error
  and checkForUpgrade() that logs a non-blocking upgrade warning
- Replace hardcoded Revyl viewer link in SessionDetailHeader with
  generic metadata-driven link system (any *_url key renders a link)
- Remove Revyl section from root README.md (docs/revyl-integration.md
  already covers it)

Code quality (per review comments):
- RevylBlazeSupport: accept TrailblazeDevicePlatform enum instead of
  raw String; add default screen dimensions per platform
- Rename getActiveSession() -> getActiveRevylSession()
- RevylExecutableTool: convert from interface to abstract class,
  implement ReasoningTrailblazeTool, rename client -> revylClient
- Merge RevylNativeLongPressTool into RevylNativeTapTool (longPress param)
- Remove RevylNativeScreenshotTool from LLM tool set
- Rename tool sets: CoreToolSet -> RevylCoreToolSet, etc.
- Fix tool naming: revyl_press_key -> revyl_pressKey
- Add ANDROID_BACK key support, iOS back clarification
- Add swipe direction validation, assert examples for LLM
- Use TrailblazeJsonInstance instead of private Json instances
- Use File.createTempFile() for screenshot paths
- Add ACTIVE_SESSION constant, REVYL_API_KEY_ENV constant
- Extract platform-to-driver-type utils in RevylDeviceService
- Fix build warnings: suppress deprecated LongPressOnElementWithText,
  remove redundant else branches in when expressions

Made-with: Cursor
Signed-off-by: Anam Hira <hira.anam49@gmail.com>
The Revyl CLI now requires --target or --x/--y for type and clear-text
commands. InputTextTrailblazeTool and EraseTextTrailblazeTool don't carry
a target field (the LLM focuses the field first via tap), so pass
"focused input field" as a default target for AI grounding.

Made-with: Cursor
Signed-off-by: Anam Hira <hira.anam49@gmail.com>
val trailblazeLlmProvider = trailblazeLlmModelList.provider
JvmLLMProvidersUtil.getEnvironmentVariableKeyForLlmProvider(trailblazeLlmProvider)
}
} + xyz.block.trailblaze.host.revyl.RevylCliClient.REVYL_API_KEY_ENV
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT: Fully qualified import is overkill.

Comment on lines +71 to +72
TrailblazeDevicePlatform.IOS -> Pair(1170, 2532)
else -> Pair(1080, 2400)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: this is as constants and document what they are.

val results = mutableListOf<RevylDeviceTarget>()
val seenModels = mutableSetOf<String>()

for (platform in listOf("android", "ios")) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could be constants for the "REVYL_PLATFORM_IOS"

Copy link
Collaborator

@handstandsam handstandsam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently, it looks like you're mapping the Trailblaze Standard tools over to Revyl. I would highly suggest using your tools directly.

Comment on lines +151 to +175
// Render external links from device metadata (convention: *_url keys become clickable links)
val metadata = sessionDetail.session.trailblazeDeviceInfo?.metadata.orEmpty()
val externalLinks = metadata.entries
.filter { it.key.endsWith("_url") && it.value.isNotBlank() }
.map { (key, url) ->
val prefix = key.removeSuffix("_url")
val label = metadata["${prefix}_label"]
?: "Open ${prefix.replace('_', ' ').replaceFirstChar { c -> c.uppercase() }}"
label to url
}
if (externalLinks.isNotEmpty()) {
val uriHandler = LocalUriHandler.current
for ((label, url) in externalLinks) {
TextButton(
onClick = { uriHandler.openUri(url) },
contentPadding = ButtonDefaults.TextButtonContentPadding,
) {
Text(
text = label,
style = MaterialTheme.typography.labelSmall,
color = MaterialTheme.colorScheme.primary,
)
}
}
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is still a lot of key lookups on the map, could we create a utility that does this to take the "deviceInfo" and come back with the "external links" data?

Comment on lines +15 to +24
data class RevylSession(
val index: Int,
val sessionId: String,
val workflowRunId: String,
val workerBaseUrl: String,
val viewerUrl: String,
val platform: String,
val screenWidth: Int = 0,
val screenHeight: Int = 0,
)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this move to :trailblaze-revyl

Comment on lines +28 to +40
data class RevylActionResult(
val action: String = "",
val x: Int = 0,
val y: Int = 0,
val target: String? = null,
val success: Boolean = true,
@SerialName("latency_ms")
val latencyMs: JsonElement? = null,
@SerialName("duration_ms")
val durationMs: Int = 0,
val text: String? = null,
val direction: String? = null,
) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this move to :trailblaze-revyl

Comment on lines +17 to +19
class RevylDeviceService(
private val cliClient: RevylCliClient,
) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this move to :trailblaze-revyl

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this be deleted?

Comment on lines +24 to +30
@Serializable
data class RevylLiveStepResult(
val success: Boolean = false,
@SerialName("step_type") val stepType: String = "",
@SerialName("step_id") val stepId: String = "",
@SerialName("step_output") val stepOutput: JsonObject? = null,
) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this move to :trailblaze-revyl

* The CLI binary must be on PATH (or set via REVYL_BINARY env var)
* and authenticated via REVYL_API_KEY or `revyl auth login`.
*/
object RevylMcpServerFactory {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can delete this.

* @property agent The Trailblaze agent that dispatches tools via CLI.
* @property platform Device platform ("ios" or "android").
*/
class RevylMcpBridge(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can delete this.

Comment on lines +12 to +49
object RevylNativeToolSet {

/** Core tools for mobile interaction -- tap, type, swipe, navigate, etc. */
val RevylCoreToolSet =
DynamicTrailblazeToolSet(
name = "Revyl Native Core",
toolClasses =
setOf(
RevylNativeTapTool::class,
RevylNativeTypeTool::class,
RevylNativeSwipeTool::class,
RevylNativeNavigateTool::class,
RevylNativeBackTool::class,
RevylNativePressKeyTool::class,
ObjectiveStatusTrailblazeTool::class,
),
)

/** Revyl assertion tools for visual verification. */
val RevylAssertionToolSet =
DynamicTrailblazeToolSet(
name = "Revyl Native Assertions",
toolClasses =
setOf(
RevylNativeAssertTool::class,
),
)

/** Full LLM tool set -- core tools plus assertions and memory tools. */
val RevylLlmToolSet =
DynamicTrailblazeToolSet(
name = "Revyl Native LLM",
toolClasses =
RevylCoreToolSet.toolClasses +
RevylAssertionToolSet.toolClasses +
xyz.block.trailblaze.toolcalls.TrailblazeToolSet.RememberTrailblazeToolSet.toolClasses,
)
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are unused, which is why you don't have your tools. We should use it somewhere

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use this in @trailblaze-host/src/main/java/xyz/block/trailblaze/host/TrailblazeHostYamlRunner.kt and other places that use the "TrailblazeDriverType.PLAYWRIGHT_NATIVE", you'll want to follow the similar pattern in the when statements.

You will need a method inside that host runner specifically like playwrightNative

Comment on lines +115 to +116
widthPixels = 0,
heightPixels = 0,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you extracted a constant for this value for iOS and for Android. Could you use it here?

- Fix --clear-first flag to always pass explicit value (CLI defaults to true)
- Add appId and buildVersionId params to startSession() for Revyl build mgmt
- Add revyl_doubleTap native tool for zoom/double-tap UI patterns
- Move Revyl data classes from trailblaze-host to trailblaze-revyl module
- Add RevylDefaults for platform-specific screen dimension fallbacks
- Wire RevylNativeToolSet for all Revyl sessions (MCP, CLI, YAML runner)
- InputTextTrailblazeTool fallback passes "text input field" target
- Add ExternalLinkUtils for viewer URL handling in session UI

Made-with: Cursor
Signed-off-by: Anam Hira <hira.anam49@gmail.com>
Made-with: Cursor
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants