Missing Trace Frame

[koftnt]

Hi all
I got Missing Trace Frame in traces UI page very often. I want to know where can I add log to find out what happen in source code?
or the codes is related to the trace frame str to show ?
thanks a lot!

[arthaud]

Hi @koftnt,
Unfortunately, this UI is currently under maintained.
I believe "Missing Trace Frame" is a problem with the tool itself producing the results (here I believe this is Mariana Trench?). This might be a bug in the implementation.
Are you using sapp and mariana trench installed from pip? It is possible that there is a mismatch between those versions.
You could try installing the previous version of Mariana Trench (try 1.0.3). You could also try to build sapp and mariana trench from source with the latest master.

[koftnt]

HI arthaud ,
Thank for you reply. I will try your advice.
I am curious what Ui you are using internally and when will be available for public?

[arthaud]

We don't plan on open sourcing our internal UI because it relies on a lot of internal services.

[koftnt]

hi arthaud
I built mariana trench from source with the latest master and sapp with the master branch.
But I got error from sapp:

sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) table issues has no column named first_instance_id
[SQL: INSERT INTO issues (id, handle, callable_id, code, status, detected_time, first_instance_id) VALUES (?, ?, ?, ?, ?, ?, ?)]
[parameters: ((3358, 'Lcom/example/myapplication/MainActivity;.onCreate:(Landroid/os/Bundle;)V:Lcom/example/myapplication/MainActivity;.launchFallback:(Landroid/content/Intent;)V:0:3:3886ec6bc93645ad', 14003, 3, 'uncategorized', 1698722093.409379, 3358), (3359, 'Lcom/example/myapplication/MainActivity;.onCreate:(Landroid/os/Bundle;)V:Lcom/example/myapplication/MainActivity$#6;.onClick:(Landroid/view/View;)V:0:1:12980548c1b721db', 14003, 1, 'uncategorized', 1698722093.409379, 3359), (3360, 'Lcom/example/myapplication/Provider;.query:(Landroid/net/Uri;[Ljava/lang/String;Ljava/lang/String;[Ljava/lang/String;Ljava/lang/String;)Landroid/database/Cursor;:Landroid/database/sqlite/SQLiteDatabase;.query:(Ljava/lang/String;[Ljava/lan:38f8ef86951646be', 14016, 4, 'uncategorized', 1698722093.409379, 3360), (3361, 'Lcom/example/myapplication/Provider;.query:(Landroid/net/Uri;[Ljava/lang/String;Ljava/lang/String;[Ljava/lang/String;Ljava/lang/String;)Landroid/database/Cursor;:Landroid/database/sqlite/SQLiteDatabase;.execSQL:(Ljava/lang/String;)V:0:4:fbe1e57d930d4601', 14016, 4, 'uncategorized', 1698722093.409379, 3361), (3362, 'Lcom/example/myapplication/WebViewActivity;.activity_lifecycle_wrapper:(Landroid/os/Bundle;)V:Lcom/example/myapplication/WebViewActivity;.onStart:()V:0:5:d39385a1fc2d3aa4', 14023, 5, 'uncategorized', 1698722093.409379, 3362), (3363, 'Lcom/example/myapplication/WebViewActivity;.activity_lifecycle_wrapper:(Landroid/os/Bundle;)V:Lcom/example/myapplication/WebViewActivity;.onStart:()V:0:2:1f9a1bf9b90c3054', 14023, 2, 'uncategorized', 1698722093.409379, 3363))]
(Background on this error at: http://sqlalche.me/e/13/e3q8)

For the pip install fb-sapp case：
When I run sapp --tool=mariana-trench analyze command .I got error like this:
Traceback (most recent call last):
File "/Users/volcanosj/opt/anaconda3/envs/mt_scan/bin/sapp", line 8, in
sys.exit(cli())
File "/Users/volcanosj/opt/anaconda3/envs/mt_scan/lib/python3.9/site-packages/click/core.py", line 1130, in call
return self.main(*args, **kwargs)
File "/Users/volcanosj/opt/anaconda3/envs/mt_scan/lib/python3.9/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/Users/volcanosj/opt/anaconda3/envs/mt_scan/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/volcanosj/opt/anaconda3/envs/mt_scan/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/volcanosj/opt/anaconda3/envs/mt_scan/lib/python3.9/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/Users/volcanosj/opt/anaconda3/envs/mt_scan/lib/python3.9/site-packages/click/decorators.py", line 84, in new_func
return ctx.invoke(f, obj, *args, **kwargs)
File "/Users/volcanosj/opt/anaconda3/envs/mt_scan/lib/python3.9/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/Users/volcanosj/opt/anaconda3/envs/mt_scan/lib/python3.9/site-packages/sapp/cli_lib.py", line 206, in analyze
pipeline.run(analysis_output, summary_blob)
File "/Users/volcanosj/opt/anaconda3/envs/mt_scan/lib/python3.9/site-packages/sapp/pipeline/init.py", line 365, in run
next_input, summary = step.run(next_input, summary)
File "/Users/volcanosj/opt/anaconda3/envs/mt_scan/lib/python3.9/site-packages/sapp/pipeline/base_parser.py", line 230, in run
self.analysis_output_to_dict_entries(
File "/Users/volcanosj/opt/anaconda3/envs/mt_scan/lib/python3.9/site-packages/sapp/pipeline/base_parser.py", line 182, in analysis_output_to_dict_entries
for typ, key, e in self._analysis_output_to_parsed_tuples(inputfile):
File "/Users/volcanosj/opt/anaconda3/envs/mt_scan/lib/python3.9/site-packages/sapp/pipeline/base_parser.py", line 131, in _analysis_output_to_parsed_tuples
for e in entries:
File "/Users/volcanosj/opt/anaconda3/envs/mt_scan/lib/python3.9/site-packages/sapp/pipeline/mariana_trench_parser_v2.py", line 470, in parse
yield from self.parse_handle(handle)
File "/Users/volcanosj/opt/anaconda3/envs/mt_scan/lib/python3.9/site-packages/sapp/pipeline/mariana_trench_parser_v2.py", line 483, in parse_handle
yield from self._parse_issues(model)
File "/Users/volcanosj/opt/anaconda3/envs/mt_scan/lib/python3.9/site-packages/sapp/pipeline/mariana_trench_parser_v2.py", line 500, in _parse_issues
(preconditions, final_sinks) = self._parse_issue_conditions(
File "/Users/volcanosj/opt/anaconda3/envs/mt_scan/lib/python3.9/site-packages/sapp/pipeline/mariana_trench_parser_v2.py", line 561, in _parse_issue_conditions
method=Method.from_json(origin),
File "/Users/volcanosj/opt/anaconda3/envs/mt_scan/lib/python3.9/site-packages/sapp/pipeline/mariana_trench_parser_v2.py", line 53, in from_json
canonical_name = method["name"]
KeyError: 'name'

[yuhshin-oss]

Regarding the error:

sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) table issues has no column named first_instance_id

It looks like the existing schema for the issues table is out of date. The following SQL command should update it (syntax might be slightly wrong):

ALTER TABLE issues ADD COLUMN `first_instance_id` bigint unsigned DEFAULT NULL;

Alternatively, if you do not need to preserve existing data in the table, you could re-create it. Run DROP TABLE issues; then run the sapp-cli as you normally would. It should create any tables that doesn't already exist.

As for the pip-install error, it just looks like it's a really old version of the sapp-cli that won't work with the latest Mariana Trench output...

[koftnt]

@yuhshin-oss
Regarding the error:
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) table issues has no column named first_instance_id

I use sapp --tool=mariana-trench analyze command to generate sapp.db file then got the error . As a result .the db file is not generated. How can I alter table or drop table issues? Shoud I alter class Issue source code in appp and delete codes related to first_instance_id column?

I try to delete the declaration first_instance_id column code in sapp/models.py. then i got error:
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) table issue_instances has no column named purge_status

[yuhshin-oss]

That's strange... if there isn't an existing sapp.db file, then it should create the tables with the correct schema. Can you double check that sapp.db really doesn't exist?

Alternatively, you could use the --dbname option to specify a .db file that you're sure doesn't exist:

sapp-cli --tool=mariana-trench --dbname mytmpfile.db analyze <analysis output dir>

You could also alter the sapp source code and delete relevant columns where these errors are showing up. I'd imagine that would be rather tedious though. The "first_instance_id" and "purge_status" columns are columns that we use internally and shouldn't be needed for viewing the results in the UI.

[arthaud]

I'm pretty sure you are using the pip fb-sapp and not fb-sapp built from master, since I see mariana_trench_parser_v2.py in the python stack trace. This won't work.

You should either use both mariana-trench and fb-sapp from pip, or build them both from master (for now, building sapp from master is broken but it will have a fix soon).