Draft: Add EUCC section to the page

[tkachyna]

• adds new EUCC section on the page
• some UX/UI visual improvements which can be discussed, they are not final

[J08nY]

Looks OK. I did not see it live.

Could you add tests? There are a bunch of tests for CC and FIPS. Replicating the same level here, would be nice.

[J08nY]

This is incorrectly rendered. It looks like you are missing the font. It is "IBM Plex Sans".

[J08nY]

Maybe this can be extracted and used more generally?

[J08nY]

This would be better as a template filter or global and implemented in Python. There are some already present, so maybe do that?

[J08nY]

Is this necessary?

[tkachyna]

upss I forgot to clean up it

[J08nY]

Also, what are the UI/UX improvements? I haven't seen them in a cursory look at the code.

Btw. nice work adding this. I hope it was not too much pain. There are no docs. Would you mind sharing some feedback on what issues (if any) you encountered or where you see potential for improvement on the site? The site code is quite messy, so feel free to comment on it.

[tkachyna]

These are the UI/UX improvements. Following a discussion with Martin, we decided that file metadata (creation/edit dates, file formats) is unnecessary. Most importantly the first section should be renamed to "Authoritative Info" instead of "CSV info" (for CC) or "Webpage info" (for FIPS) so it is immediately clear to the user. Then it is necessary to visually separate the second section, the "Nonauthoritative" part, which contains PDF extractions and heuristics. These changes are not yet final, but I am quite satisfied with them so far. Jakub Borsky will try to deploy this on some virtual machine and Mariia will take a look at it from the UX/UI perspective. :) There is still some work to do, I want to improve scraping of the contacts a reorganize a code little bit.

[J08nY]

Ok, I get this. Few comments:

• Calling it "Authorithative info" and "Nonauthoritative info" feels wrong, for several reasons. For Common Criteria, the "Authoritative Info" part is not even true, as the actual scheme websites are the authoritative ones and not Common Criteria Potral where we get our stuff from. For sure, it is a a trustworthy source to a degree. Their data is also full of typos, errors and just all around nonsense, similar to how some of our heuristics are messy. Can we figure out a better naming?
• I like the table format. However, the heuristics summary was up-top for a reason. There is value in our heuristics and they are quite good most of the time. I mean, if we hide away all of the stuff we painfully extracted, then the sec-certs web is a glorified excel renderer for the CC portal CSV. Specifically, someone (I guess Petr) previously asked for the summary with the cert ID to be prominently added to the top of the page, so lets not go in circles.
• Re: file metadata, quite often this contains the certificate ID, which the user is interested in. With the changes done currently the ID is even more hidden very deep in the page. Please no. Lets collapse the info for example, but lets keep it around.

I feel like we are going at it a bit backwards, instead of improving our heuristics such that they are more accurate and can be trusted, we are hiding them away and putting big scary disclaimers on them. I understand the need for clarity and data provenance, but this just reduces the whole point of having the sec-certs site at all. @mukrop @jborsky @MaryBak Could we chat about this a bit?

[tkachyna]

• I think there were some ideas on how to name these two better. If I remember correctly, for Authoritative info, it was something like "Data extracted from the official page" (just a bit shorter), and for the second one, "Data that we processed" Yeah we definitely need to brainstorm these some more. But it would be nice to unify these names across EUCC, CC and FIPS.
• I get why there is a Heuristics section at the top. In CC, there is no certificate ID in either the CSV or HTML, so we have to scrape it from the PDFs and use heuristics to pick the best fitting value. However for EUCC we are scraping the cert ID directly from the metadata table on the ENISA page (next to a label that says "this is the ID"). This makes using heuristics for that ID feel a bit redundant now. These changes only apply to EUCC for now (unless we decide otherwise in the future) :).
• Ok, I did not know that there could be some valuable info, most of the time it was not that useful for me. I reverted it but made a little change as you suggested. It is inside a collapsible.

[jborsky]

@J08nY

I feel like we are going at it a bit backwards, instead of improving our heuristics such that they are more accurate and can be trusted, we are hiding them away and putting big scary disclaimers on them. I understand the need for clarity and data provenance, but this just reduces the whole point of having the sec-certs site at all. @mukrop @jborsky @MaryBak Could we chat about this a bit?

Of course, I’d like to hear your thoughts. Just to clarify the current state: this version from Tadeas is neither the final version nor one we have agreed upon. We saw it for the first time last Thursday, and we agreed that once Tadeas pushes this branch, I will deploy it on a test VM. This will let Mariia and the rest to see it easily live and provide feedback. Once everyone has had a chance to look it over, we'll set up a meeting together to discuss it. So there will be plenty of chances to make changes before it's merged.

[mukrop]

@tkachyna, I'd generally suggest to only keep adjustments to EUCC sections in this PR and if we're doing other changes that would include other parts of the site, let's have them in a separate PR with an appropriate discussion.

Although I'm slightly skeptical of the idea that the site visitor read the file metadata to determine the certificate ID, I do agree with Jano that collapsing things is probably a better idea now than fully deleting them. Without any user-testing any adding/removing is a but of a guesswork, but I believe Mariia has user-testing planned soon :-).

@J08nY's comment on the (non-)prominence of heuristics/our metadata is very relevant. Though I'm afraid getting it adjusted well will not be too easy and will include some more discussion.

[tkachyna]

Before making any changes to the current PR, I will save the current state into another branch and then modify this one.

But first I want to agree with you on the following, what will be added (or modified) in the UI:

• add webpage metadata from the ENISA page with the title "Webpage metadata" (as it is currently in the FIPS section)
• I will keep the heuristics table at the top, but due to the length of the data in the metadata table (from the ENISA page), there is no way to place them next to each other. The only solution is to stack them vertically
• I lean towards the opinion to keep the file metadata in the collapsible (as it was implemented in the last commit)
• In the images below, I want to personally keep the gray background and the paddings from the left as it is in this PR (ofc the red parts will be removed), these changes will only apply to the EUCC (not CC or FIPS).

(the use with caution (heuristics) alert will remain as it was already present it is not part of my change)

[mukrop]

Sounds reasonable to me, but I'd prefer to see it as a dev preview (I believe @jborsky already has it set up). Then we can all have a click-through and either have a short call or exchange opinions in the chat.

[tkachyna]

I lI will try to make these changes tomorrow and the day after so that we can discuss them after thursday weekly. I would prefer a call as this could be a pretty long discussion.