ReFLAIR

ReFLAIR (Reflow Fault Localization using AI-based Responsive analysis) is a Selenium- and LLM-assisted workflow for checking whether a reflowed webpage preserves the information and functionality available in its original desktop version.

The repository contains paired webpage snapshots, ground-truth annotation data, and Python scripts that compare original pages (O_*) against reflowed pages (R_*). The core pipeline extracts visible text, icon/image widgets, and actionable elements, summarizes their behavior, compares the original and reflowed versions, and writes JSON reports for missing or inconsistent widgets.

Repository Layout

ReFLAIR/
├── Code/
│   ├── requirements.txt
│   └── src/
│       ├── ReFLAIR/
│       ├── baselines/
│       └── scripts/
├── Ground Truth/
└── Subject/
    ├── New_Subject/
    └── Subject/

Data Conventions

Each subject is represented by a row in Code/src/ReFLAIR/subject_folder.csv:

webpage,base_path,folder_name
example.com,/absolute/path/to/subject/files,Example Page

For each row, base_path should contain paired HTML files named:

O_<folder_name>.html or O_<folder_name>.htm
R_<folder_name>.html or R_<folder_name>.htm

O_ files are the original/desktop pages. R_ files are the reflowed/mobile pages. Replace placeholder values such as <Absolute Path to the Subject> with real paths before running the pipeline.

Environment

The scripts load WebTesting.env from the current working directory, so run the main commands from:

cd Code/src/ReFLAIR

Configure WebTesting.env:

OPENAI_API_KEY=
GEMINI_API_KEY=
ORIGINAL_X=1280
ORIGINAL_Y=1024
REFLOW_X=320
REFLOW_Y=1024

Only one model key is needed for the corresponding pipeline:

• OpenAI scripts use OPENAI_API_KEY.
• Gemini scripts under gemini/ use GEMINI_API_KEY.

Do not commit API keys.

Installation

Use Python 3.10 or newer. Firefox must be installed because the pipeline uses Selenium's Firefox WebDriver.

cd Code
python -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -r requirements.txt

Then move to the script directory:

cd src/ReFLAIR

Running the OpenAI Pipeline

--ex_subj_id is the 1-based row index in subject_folder.csv.

1. Extract and summarize widgets from the original page:

   python main.py --ex_subj_id 1

   This creates an output directory named from EXPERIMENT_NAME and a timestamp, for example TEST0_20260705-090000.

2. Extract and summarize widgets from the reflowed page:

   python mainR.py --ex_subj_id 1 --ex_out_path TEST0_20260705-090000

3. Compare original and reflowed summaries:

   python mainR_check.py --ex_subj_id 1 --ex_out_path TEST0_20260705-090000

   This writes issue_not_found.json.

4. Re-check unresolved issues on the live page state:

   python firefox_agent.py --ex_subj_id 1 --ex_out_path TEST0_20260705-090000
   python firefox_agent2.py --ex_subj_id 1 --ex_out_path TEST0_20260705-090000

5. Build the final manual review bundle:

   python manual_annotation.py --ex_subj_id 1 --ex_out_path TEST0_20260705-090000

   This writes 0_final_manual_data.json.

Several scripts pause with an interactive prompt after loading a page. Follow the prompt in the terminal before continuing.

Running the Gemini Pipeline

The Gemini variants mirror the OpenAI flow and live under Code/src/ReFLAIR/gemini. Set GEMINI_API_KEY in WebTesting.env before using these scripts. Run them from Code/src/ReFLAIR so WebTesting.env and subject_folder.csv are found:

python main.py --ex_subj_id 1
python mainR.py --ex_subj_id 1 --ex_out_path TEST0_20260705-090000
python gemini/mainR_check.py --ex_subj_id 1 --ex_out_path TEST0_20260705-090000
python gemini/firefox_agent.py --ex_subj_id 1 --ex_out_path TEST0_20260705-090000
python gemini/firefox_agent2.py --ex_subj_id 1 --ex_out_path TEST0_20260705-090000

Outputs

The pipeline writes JSON and screenshot artifacts into the experiment output directory. Common intermediate files include:

• iconimage.json, text_xpath.json, actionable_xpath.json
• function_summary_text.json
• function_summary_iconimage.json
• function_summary_actionable.json
• R_function_summary_text.json
• R_function_summary_iconimage.json
• R_function_summary_actionable.json
• issue_not_found.json
• 2_issue_not_found.json
• 2_final_report.json

Final output includes:

• 0_final_manual_data.json
• execution_time.txt

Screenshot artifacts are stored in folders such as function_summarization/, R_function_summarization/, tmp/, tmpr/, and 3R_free_exploration/.

Utility Scripts

Count widgets in configured subjects:

python widget_cnt.py

The result is written to widget_count_results.csv.

The baselines/ scripts contain baseline LLM comparisons. Some paths in those files are placeholders and should be configured before use.

Notes

• Prefer absolute paths in subject_folder.csv; some scripts construct file URLs directly from base_path.
• Keep WebTesting.env local and do not commit API keys.