feat: Revamp vulnerability page

[ppkarwasz]

This pull request proposes a new design for the Apache Solr security page, as discussed on dev@solr, and introduces support for richer metadata in vulnerability-related articles.

Key Changes

• Full YAML front matter support in Markdown articles

  • The standard PyMarkdown MetaExtension only parses flat YAML front matter.
  • This change introduces YamlFrontMatterExtension, which supports complex YAML documents inside front matter.

• Defined YAML structure for vulnerability articles

  • Applies to vulnerabilities in Apache Solr itself or in its dependencies.
  • Contains sufficient information to generate a CycloneDX VEX file.

• Enhanced security page

  • Lists vulnerabilities by Solr minor version along with their status (in_triage, not_exploitable, affected).
  • Displays detailed vulnerability articles, with layout variations depending on the type and assessment state.

Work in progress

I am still working on generating CycloneDX documents (VDR and VEX) from the Markdown articles.
However, early feedback from the maintainers and the community would be valuable to ensure I am heading in the right direction before completing the implementation.

Preview

1. YAML front matter

The YAML front matter of the articles looks like this:

cve: CVE-2024-51504
jira: SOLR-17809
severity: important
category:
  - solr/vex
vulnerable_component:
  name: zookeeper
  versions:
    - introduced: 3.9.0
      last_affected: 3.9.2
impacted_component:
  name: Apache Solr
  versions:
    - introduced: 9.4.0
      fixed: 9.9.0
analysis:
  state: not_affected
  justification: requires_configuration
title: "Apache ZooKeeper: Authentication bypass with IP-based authentication in Admin Server"

2. Vulnerabilities Table

Lists vulnerabilities by Solr minor version with their current status.

Show screenshot

3. Vulnerability Articles

a. Vulnerability in Solr itself

Show screenshot

b. VEX entry in triage

Show screenshot

c. VEX entry assessed

Show screenshot

[janhoy]

@ppkarwasz I just realized that my security-page restructuring in #168 will be in total conflict with the work in this PR. I apologize for not checking before I started that work. Also I just merged an overhaul of python dependencies, added a lockfile, bumped Pelican to latest version etc, which may also affect some of the work here, wrt Pelican plugin.

Again, sorry for not coordinating up front. I hope you will be able to fit some of these improvements on top of the new security pages. There will be a separate VEX page etc.

[epugh]

@ppkarwasz I hope it's okay I took a stab at updating your PR with the latest changes?

[epugh]

Here is the table view with a new "title" column tjhat is clickable:

Each title is now linked to a row in the vex.html file:

[epugh]

@ppkarwasz @janhoy I hope my work is useful on this PR... I took the original PR, and bent it in the direction of how can we use these vex files... I wanted our front matter to be compatible with the cyclonedx format, and not to be going beyond what CycloneDX VEX supports. So I removed vulnerable_component, impacted_component, and severity.

I think those other annotations could be really useful, but I feel like we are trying to do too much in one PR. Now, this PR would get us to a long term fomrat of using markdown, a nice detail page, and then if we want to start adding additional metadata we could.

If you all agree, then we can move this out of DRAFT!

[janhoy]

I fail to visualize to myself how this will fully look like.

But the idea to maintain the list of (non) vulnerable dependency CVEs in Yaml/MD sounds like an improvement. Question is how we'll make sure we maintain it. Same issue as before.

This could be a good task for an agent perhaps. Each week, grab list of dependency CVEs from our dependencies, filter on the highest severities (some threshold), check out solr codebase, do an assessment on whether Solr is vulnerable or not. Make a draft solr-site PR with the verdict. And if we are vulnerable, send an email to security@ list..

[ppkarwasz]

@epugh: go ahead and improve this PR.

There is one part of the PR that can be simplified: the yaml_front_matter extension should be replaced with the yaml-metadata external extension, so we don't have to maintain it.

Regarding the generation of VEX file, this should absolutely be done by an agent. PR #163 is sooo last year… These days we should probably have a workflow that:

1. Downloads the source code of all dependencies on the path Solr -> vulnerability. This is still not a trivial task (the <scm> metadata in the POMs is often incorrect), but we have a small manually validated database in callgraph-metadata. Later on we could replace it with something better: AboutCode and OSS Review Toolkit each have their own way to find the source code.
2. The callgraph metadata can be used to generate reachable paths like in Analyse impact of CVE-2024-21742 on Solr 9.10.0 vex-generation-toolset/solr-site#1.
3. Everything could be fed to an agent to validate each path and determine, when it is exploitable.

[epugh]

@ppkarwasz Great suggestion! That removed a couple of files!

@janhoy if you can give this a review, then we can at least get to movign with our .md format for vex issues, and then figure out the agent workflows in a future PR.

[Copilot]

Pull request overview

This PR revamps the Solr security/VEX content pipeline and presentation by moving dependency-vulnerability entries from a static JSON input into Markdown articles with YAML front matter, then rendering those entries via updated Pelican templates and styling.

Changes:

• Replace vex-input.json with per-entry Markdown VEX articles (content/solr/vex/*.md) using YAML front matter validated against a schema.
• Update the vex Pelican plugin to parse/validate front matter, generate solr.vex.json, and provide table data to templates.
• Add a dedicated vex.html page and update the dependency CVE table to link to individual VEX articles, with new status styling.

Reviewed changes

Copilot reviewed 49 out of 51 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
vex-input.json	Removes the old static JSON VEX input (migrated to Markdown articles).
themes/solr/templates/vex.html	New template for rendering VEX article pages and details.
themes/solr/templates/security-dependency-cves.html	Updates dependency CVE table to link to VEX articles and adjusts ordering/columns.
themes/solr/static/css/base.css	Adds status badge styling and :target scroll offset for anchor navigation.
requirements.txt	Adds dependencies for YAML front matter parsing/validation.
requirements.in	Adds pelican-yaml-metadata and strictyaml inputs.
plugins/vex/vex.py	Reads VEX Markdown + YAML, validates front matter, generates CycloneDX VEX JSON, sets slugs/anchors.
plugins/vex/tests/init.py	Adds test package placeholder (license header).
plugins/vex/schema/vex-input.schema.json	Removes obsolete schema for the deleted JSON input format.
plugins/vex/schema/vex_article.schema.yaml	Adds schema for VEX article YAML front matter.
plugins/vex/schema/bom-1.4.schema.json	Removes bundled CycloneDX 1.4 schema (migration to 1.6 schema file).
pelicanconf.py	Enables yaml_metadata plugin and adds Jinja environment extensions.
content/solr/vex/2025-08-02-cve-2024-7254.md	New VEX article entry (YAML front matter + description).
content/solr/vex/2025-07-25-cve-2024-51504.md	New VEX article entry with detailed assessment text.
content/solr/vex/2025-02-26-cve-2024-6763.md	New dependency VEX entry (Jetty).
content/solr/vex/2025-01-26-cve-2025-24814.md	New VEX entry for a Solr CVE article.
content/solr/vex/2024-01-12-cve-2023-51074.md	New dependency VEX entry including GHSA id.
content/solr/vex/2022-12-14-cve-velocity-tools.md	New dependency VEX entry (Velocity Tools).
content/solr/vex/2022-12-14-cve-tika-core.md	New dependency VEX entry (Tika core catch-all).
content/solr/vex/2022-12-14-cve-2022-42889.md	New dependency VEX entry (commons-text).
content/solr/vex/2022-12-14-cve-2022-39135.md	New VEX entry marked exploitable (Calcite).
content/solr/vex/2022-12-14-cve-2022-33980.md	New dependency VEX entry (commons-configuration2).
content/solr/vex/2022-12-14-cve-2022-25168.md	New dependency VEX entry (hadoop-common).
content/solr/vex/2022-12-14-cve-2021-45105.md	New dependency VEX entry (Log4j multi-CVE).
content/solr/vex/2022-12-14-cve-2021-44832.md	New dependency VEX entry (Log4j JDBCAppender CVE).
content/solr/vex/2022-12-14-cve-2021-33813.md	New dependency VEX entry (JDOM).
content/solr/vex/2022-12-14-cve-2020-27223.md	New dependency VEX entry (Jetty symlink-related).
content/solr/vex/2022-12-14-cve-2020-27218.md	New dependency VEX entry (Jetty GzipHandler-related).
content/solr/vex/2022-12-14-cve-2020-13955.md	New dependency VEX entry (avatica/calcite).
content/solr/vex/2022-12-14-cve-2019-16869.md	New dependency VEX entry (netty-all).
content/solr/vex/2022-12-14-cve-2019-10241.md	New dependency VEX entry (Jetty multi-CVE).
content/solr/vex/2022-12-14-cve-2019-10086.md	New dependency VEX entry (commons-beanutils).
content/solr/vex/2022-12-14-cve-2018-8088.md	New dependency VEX entry (slf4j).
content/solr/vex/2022-12-14-cve-2018-1471.md	New dependency VEX entry (simple-xml).
content/solr/vex/2022-12-14-cve-2018-1335.md	New dependency VEX entry (tika-core.1.17).
content/solr/vex/2022-12-14-cve-2018-10237.md	New dependency VEX entry (carrot2-guava).
content/solr/vex/2022-12-14-cve-2018-10237-guava.md	New dependency VEX entry (guava catch-all).
content/solr/vex/2022-12-14-cve-2018-1000632.md	New dependency VEX entry (dom4j).
content/solr/vex/2022-12-14-cve-2018-1000056.md	New dependency VEX entry (junit).
content/solr/vex/2022-12-14-cve-2017-15718.md	New dependency VEX entry (hadoop-auth).
content/solr/vex/2022-12-14-cve-2017-15095.md	New dependency VEX entry (jackson-databind multi-CVE).
content/solr/vex/2022-12-14-cve-2017-14952.md	New dependency VEX entry (icu4j).
content/solr/vex/2022-12-14-cve-2017-14868.md	New dependency VEX entry (org.restlet).
content/solr/vex/2022-12-14-cve-2016-6809.md	New dependency VEX entry (vorbis-java-tika multi-CVE).
content/solr/vex/2022-12-14-cve-2015-5237.md	New dependency VEX entry (protobuf-java).
content/solr/vex/2022-12-14-cve-2014-7940.md	New dependency VEX entry (lucene-analyzers-icu multi-CVE).
content/solr/vex/2022-12-14-cve-2014-0114.md	New dependency VEX entry (commons-beanutils).
content/solr/vex/2022-12-14-cve-2012-2098.md	New dependency VEX entry (commons-compress multi-CVE).
content/solr/vex/2022-12-14-cve-2012-0881.md	New dependency VEX entry (xercesImpl).
content/pages/vex.md	Adds a Pelican page that renders the VEX template at vex.html.

Comments suppressed due to low confidence (1)

themes/solr/templates/security-dependency-cves.html:37

• GHSA identifiers in the dependency CVE table are currently rendered as plain text. Since the VEX input supports GHSA IDs, these should be linkified to the corresponding GitHub Advisory pages (similar to CVE→NVD links).

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[epugh]

Okay, I would love to get a +1 from someone!

[janhoy]

Let’s merge to main first so we see the staging site

[epugh]

I've merged, and what I expected to see on https://solr.staged.apache.org/security-dependency-cves.html is there... What is the process to get it to the production branch?

[janhoy]

Merge main --> production, using a merge-commit not a squash.