Adding technology classes

[atpham88]

Summary

This PR:

• Removes startyear switch and instead establishes startyear based on yearset
• Adds algorithm to assign pv, wind, and geothermal tech classes to NEMS units
• Fixes a few issues that cause missing existing capacities in main
• Adds an option to specify a flexible start year, instead of 2010
  • Although this option is added in this PR, enabling and testing for start year later than 2010 is in a separate PR.

Technical details

These changes are made in this PR:

• process_unitdata.py: Read unitdata.csv from inputs_case and process it to assign sc_point_pids, capacity factors/mean temps to pv, wind, and geothermal units. This file is run immediately after copy_files.py. This allows us to remove all the mapping of NEMS units to sc_point_gids in the ReEDS_input_processing repo.
• WriteHintage.py: Allow start year to be flexible instead of 2010.
• writecapdat.py: Add mappings to solar, wind, geothermal resource class are based on the resource quality (capacity factors for wind and solar and temperature for geothermal) of the technology as it comes from reV.
• c_supplymodel.gms
  • pcat set is removed from the model: The equations eq_force_prescription_power and eq_force_prescription_energy, along with the variables EXTRA_PRESCRIP and EXTRA_PRESCRIP_ENERGY, are currently indexed by pcat. Since the prescribed generator classes and vintages are now defined in writecap.py, the pcat index is no longer needed in these equations or variables. Accordingly, the equations are revised so that the investment (INV) of prescribed generators—indexed by (i, newv, r, t)—is equal to the sum of the prescribed build values and the additional prescribed investments (EXTRA_PRESCRIP) for those same (i, newv, r, t) combinations.
  • Integrate prescribed retirements when the prescribed facilities retires before their life time: In the NEMS database, the retirement year of some facilities occurs before the end of their expected lifetimes. To accurately capture these early retirements for prescribed facilities, the prescribed retirement values are subtracted from total investments when calculating facility capacities in eq_cap_new_noret and eq_cap_energy_new_noret. Although this implementation follows the same logic as m_capacity_exog, it explicitly captures prescribed generator builds in the INV variables in the current version of the model.
• hourlize:
  • Remove scripts that generate exog_cap_{tech}.csv and those are associated with existing_sites files as they are no longer needed.
• geothermal_classification.csv is not generated in hourlize but taken from https://pages.github.nrel.gov/ReEDS/ReEDS-2.0/model_documentation.html#technical-resource-potential.
• Update checks on prescribed builds and existing capacity to ensure feasibility with resource constraints.

Issues resolved

• Corrected prescribed_build to capture all capacity in unitdata.csv. Currently some prescribed coal capacities are missing due to coalolduns is missing from prescribed_build categories https://github.com/ReEDS-Model/ReEDS/blob/main/reeds/input_processing/writecapdat.py#L79
• Corrected missing capacity in capnonrsc. This is due to due to the comment='#' option when read in unitdata.csv, which ignores all the data after unit ID of the rows where unit ID has # character. So when capnonrsc is calculated by group summing from unitdata.csv, some of the categories to sum over are nan and those rows are omitted from the sum.

Validation, testing, and comparison report(s)

Total prescribed builds and exogenous caps are generally the same with main, but spatial distribution is slightly different:
Comparison of prescribed builds (prescribed_build) vs. main (noncumulative_prescriptions) in 2010 for UPV - Total UPV capacity matches and matches with the ReEDS-NEMS plant database but there's some difference in spatial distribution.

Comparison of prescribed builds (prescribed_build ) vs. main (noncumulative_prescriptions) in 2015 for UPV: Total UPV capacity in this branch is slightly lower but it matches exactly with NEMS.

Comparison of prescribed builds (prescribed_build ) vs. main (noncumulative_prescriptions) in 2020 for UPV: Total UPV capacity in this branch matches with main and matches exactly with NEMS. Spatial distribution varies again as expected.

Comparison of prescribed builds (prescribed_build ) vs. main (noncumulative_prescriptions) in 2023 for UPV: Total UPV capacity in this branch matches with main and matches exactly with NEMS. Spatial distribution varies again.

Comparison of prescribed builds (prescribed_build ) vs. main in 2010 for wind-ons - Wind-ons capacity matches exactly:

Comparison of existing cap (exog_upv) vs. main for UPV - Existing UPV capacity is slightly higher in this branch but matches exactly with NEMS:

Comparison of existing cap (exog_wind-ons) vs. main for wind-ons - Existing wind-ons capacity matches exactly:

Comparison report (full here
Inputs comparison is here

• Increase in coal capacity due to capturing coalolduns prescribed capacity not captured in main
• Slight decrease in solar due to lower prescribed pv capacity in this branch compared to main, but this branch's prescribed pv capacity matches NEMS (see above)

• Increase in csp generation since main does not have any csp generation (fails to convert upv generation to csp generation)

Checklist for author

Details to double-check

• Charge code provided for review
• Included comparison reports for appropriate test cases
• Documentation updated if necessary
• If input data added/modified:
  • Dollar year recorded and converted to 2004$ for GAMS
  • Timeseries are in Central Time
  • Units are specified
  • Preprocessing steps have been documented and committed to ReEDS_Input_Processing
  • New large data files handled with .h5 instead of .csv
  • If spatially resolved inputs are modified, the following visualizations for each file are included in the PR description (time-averaged if the inputs are time-resolved):
    • Map of absolute values before
    • Map of absolute values after
    • Map of differences: (after - before) or (after / before)
  • If entries are added/removed/changed in the EIA-NEMS unit database:
    • Changes have been committed to ReEDS_Input_Processing
    • hourlize/resource.py was rerun to regenerate the existing/prescribed VRE capacity data
• Code formatting standardized
• Reusable functions used where possible instead of copy/pasted code

General information to guide review

• Zero impact on results of default case
• No large data file(s) added/modified
• No substantive impact on runtime for full-US reference case
• No substantive impact on folder size for full-US reference case
• No change to process flow (runbatch.py, d_solve_iterate.py)
• No change to code organization
• No change to package requirements (environment.yml or Project.toml)

Did you use LLM tools (chatbot or copilot) in the preparation of this PR? If so, describe how

No

Tag points of contact here if you would like additional review of the relevant parts of the model

[patrickbrown4]

I haven't gone through the whole thing yet and will take a closer look next week, but some initial thoughts in the meantime. Happy to discuss if helpful (including whether to try some of the suggested changes here or in a followup PR). Thanks!

[patrickbrown4]

Thanks for all your work on this one An. A few suggested cleanups but nothing requiring a new set of full-US comparisons, so all set from my end.

(The one caveat is that I didn't go through the geothermal changes in b_inputs.gms super closely, so I'll defer to you on those.)

[patrickbrown4]

Can the changes to this file be reverted? It doesn't look like any functionality has changed.

[atpham88]

What do you think about cleaning up/reverting this file in the follow up PR? I don't want to mess anything up now. I added this as an item to address in the follow up PR #158

[patrickbrown4]

Also, I think we discussed earlier, but just as a quick reminder - since the EIA-NEMS database and supply curve files went through a few iterations earlier on that ended up being reverted, and they're some of the biggest git-tracked files, please select "Squash and merge" when merging to avoid keeping those big changes in the history. Thanks!

[atpham88]

Thanks for all your work on this one An. A few suggested cleanups but nothing requiring a new set of full-US comparisons, so all set from my end.

(The one caveat is that I didn't go through the geothermal changes in b_inputs.gms super closely, so I'll defer to you on those.)

Thanks Patrick. the geothermal changes here are only a partial fix to the geothermal prescribed build issue we have been seeing, so after this PR is merged in, we will revisit it in the follow up PR along with Yunzhi's PR #94 for a full fix.