Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
171 commits
Select commit Hold shift + click to select a range
9bfa8b0
renamed dat to catch. Ran for 2024 data update.
ClaireGonzales-NOAA Feb 21, 2025
bc5c445
update capitalization of input data column names
ClaireGonzales-NOAA Feb 21, 2025
3bbf2e4
Updating more capitalization of column names
ClaireGonzales-NOAA Feb 21, 2025
3e58c56
ggplot formatting
ClaireGonzales-NOAA Feb 21, 2025
405a627
figure sizing and adjusting goa filter to be >= 12 years
ClaireGonzales-NOAA Feb 24, 2025
4fef6a6
Question about strata vs stratum. Additional axis adjustments
ClaireGonzales-NOAA Feb 24, 2025
4bfe693
test
ClaireGonzales-NOAA Feb 26, 2025
e4d0051
test 2
ClaireGonzales-NOAA Feb 26, 2025
95de328
test 3
ClaireGonzales-NOAA Feb 26, 2025
f3f56f7
test_end
ClaireGonzales-NOAA Feb 26, 2025
0ff85bf
Edit "Bering Sea Combined" to be "Eastern and Northern Bering Sea"
ClaireGonzales-NOAA Mar 3, 2025
6e9c474
Edits to gulf workflow
ClaireGonzales-NOAA Mar 7, 2025
8e5f3f5
Incorporation of workflow for the gulf
ClaireGonzales-NOAA Mar 17, 2025
73f9a6e
streamlined issue 1
ClaireGonzales-NOAA Mar 19, 2025
db9fa66
Streamlined through line 1181 (this task still in progress)
ClaireGonzales-NOAA Mar 19, 2025
e05c090
updates to code streamlining still in progress
ClaireGonzales-NOAA Mar 20, 2025
d995a84
cleaning
ClaireGonzales-NOAA Apr 1, 2025
ba207c2
gmex code edits
ClaireGonzales-NOAA Apr 3, 2025
4a32a91
gmex edits
ClaireGonzales-NOAA Apr 4, 2025
0503c07
Remove Canadian compile block (archive for future re-integration)
ClaireGonzales-NOAA Apr 4, 2025
ec9591a
remove gmex_original code chunk
ClaireGonzales-NOAA Apr 9, 2025
44733fd
edit of gmex code
ClaireGonzales-NOAA Apr 9, 2025
7f3f198
final push for gmex code
ClaireGonzales-NOAA Apr 9, 2025
ef2410d
in progress: Working through some API issues
ClaireGonzales-NOAA Apr 14, 2025
651012c
Working through edits to catch API (in progress)
ClaireGonzales-NOAA Apr 15, 2025
e990b2f
small edit to annotations
ClaireGonzales-NOAA Apr 18, 2025
a72f01f
Draft of species persistence code
ClaireGonzales-NOAA Apr 18, 2025
a40e3d1
Adding annotation
ClaireGonzales-NOAA Apr 21, 2025
fa8772e
editing to include all regions (still in progress)
ClaireGonzales-NOAA Apr 22, 2025
3b8a79d
Adding code to save alldata RDS during final steps
ClaireGonzales-NOAA Apr 22, 2025
c39f474
code built for percentile wtcpue
ClaireGonzales-NOAA Apr 23, 2025
7da243d
Building nested forloop for net biomass gain/loss (still in progress)
ClaireGonzales-NOAA Apr 24, 2025
d01df7a
calculation for cumulative biomass (completed)
ClaireGonzales-NOAA Apr 24, 2025
ca7ed93
first attempt at viz
ClaireGonzales-NOAA Apr 24, 2025
723d3ea
bringing in hawaii data for processing in species persistence script
ClaireGonzales-NOAA Apr 25, 2025
4cde469
including hawaii data
ClaireGonzales-NOAA Apr 25, 2025
7f5ef1a
remove an annotation
ClaireGonzales-NOAA Apr 25, 2025
de09e5e
temporary adjustments!! Undo later
ClaireGonzales-NOAA Apr 28, 2025
17de083
fixing the region label for Hawaii
ClaireGonzales-NOAA Apr 29, 2025
718560f
example visualizations added to end of the script (temporary)
ClaireGonzales-NOAA Apr 29, 2025
ba4265e
Creating an .Rmd to build and circulate multiple options for represen…
ClaireGonzales-NOAA Apr 29, 2025
55c7d94
Progress on percentile and haul number calculations
ClaireGonzales-NOAA May 1, 2025
9214692
update gitignore to include htmls
ClaireGonzales-NOAA May 1, 2025
e4492b3
Processing for percentile and haul counts complete.
ClaireGonzales-NOAA May 1, 2025
b329f21
heatmap visualization
ClaireGonzales-NOAA May 2, 2025
3f18aa8
remove example chunk and scaling
ClaireGonzales-NOAA May 2, 2025
e9e313b
edits
ClaireGonzales-NOAA May 2, 2025
9287388
adding jpgs to the gitignore
ClaireGonzales-NOAA May 2, 2025
f4981cb
viz updates
ClaireGonzales-NOAA May 5, 2025
d44a254
processing updates
ClaireGonzales-NOAA May 6, 2025
b4b6a45
viz updates
ClaireGonzales-NOAA May 6, 2025
9bb3478
Final viz!
ClaireGonzales-NOAA May 6, 2025
7e81fba
issues knitting the ggplotly chunks
ClaireGonzales-NOAA May 6, 2025
f122039
Draft completed
ClaireGonzales-NOAA May 6, 2025
f1a7400
Layout edits
ClaireGonzales-NOAA May 6, 2025
9fe315b
Removed some unneccessary notes
ClaireGonzales-NOAA May 13, 2025
01b7426
In progress. Added Absence/Presence to table 1 and integrated for_loo…
ClaireGonzales-NOAA May 13, 2025
bb94480
in progress. Adding col and ext tally.
ClaireGonzales-NOAA May 14, 2025
cb9b6be
col and ext columns completed
ClaireGonzales-NOAA May 15, 2025
77d90e4
In progress. knitting viz for all surveys (via for_loop())
ClaireGonzales-NOAA May 15, 2025
46ab447
knittable vizs
ClaireGonzales-NOAA May 16, 2025
575e0fb
still in progress of definitions
ClaireGonzales-NOAA May 16, 2025
86c1aff
function added that adds a trend category
ClaireGonzales-NOAA May 20, 2025
bb52629
to streamline
ClaireGonzales-NOAA May 21, 2025
7331942
edits
ClaireGonzales-NOAA May 21, 2025
ac31e7b
beginning brainstorm overflow
ClaireGonzales-NOAA May 21, 2025
88e1bf5
rdata
ClaireGonzales-NOAA May 22, 2025
d2604f1
edits
ClaireGonzales-NOAA May 22, 2025
12ecce1
rename
ClaireGonzales-NOAA May 22, 2025
203881c
edits
ClaireGonzales-NOAA May 22, 2025
c1bcf5d
small edits
ClaireGonzales-NOAA May 23, 2025
3385e05
streamlining to focus on alt 4
ClaireGonzales-NOAA May 27, 2025
c930534
writing csv for dev team
ClaireGonzales-NOAA May 28, 2025
d9836bb
export csv for dev team
ClaireGonzales-NOAA May 28, 2025
f459b65
redo csv chunk
ClaireGonzales-NOAA May 28, 2025
a6cfa32
In progress. Working on species flag code
ClaireGonzales-NOAA Jun 2, 2025
da71bbc
cleaning
ClaireGonzales-NOAA Jun 4, 2025
47dfb9a
cleaning chunks and flagging function
ClaireGonzales-NOAA Jun 4, 2025
e69077d
update csv name
ClaireGonzales-NOAA Jun 4, 2025
ea3d4c0
compilation of flagged species
ClaireGonzales-NOAA Jun 4, 2025
295b295
streamlining
ClaireGonzales-NOAA Jun 5, 2025
01e21f5
cleaning
ClaireGonzales-NOAA Jun 5, 2025
adfa70b
streamlining
ClaireGonzales-NOAA Jun 5, 2025
01191a8
rename
ClaireGonzales-NOAA Jun 6, 2025
05820d8
rename
ClaireGonzales-NOAA Jun 6, 2025
1c1d314
added summary code for tech report table
ClaireGonzales-NOAA Jun 18, 2025
2c2df77
annotations and updates to BFISH columns
ClaireGonzales-NOAA Jun 18, 2025
fa46b2c
update to annotations
ClaireGonzales-NOAA Jun 20, 2025
db3ef49
Annotation edits
ClaireGonzales-NOAA Jun 23, 2025
222a333
update to 'write.csv()' url
ClaireGonzales-NOAA Jun 24, 2025
ffbc3c1
include all species codes
ClaireGonzales-NOAA Jun 25, 2025
5870f8c
edits to code and annotations added
ClaireGonzales-NOAA Jun 26, 2025
baf7df3
added annotation
ClaireGonzales-NOAA Jun 26, 2025
9c8eae9
annotations
ClaireGonzales-NOAA Jun 27, 2025
c4ec023
logging this for potential review later
ClaireGonzales-NOAA Jul 1, 2025
5c8abfb
reverting
ClaireGonzales-NOAA Jul 1, 2025
0c7f67e
adjusting filters for stratum
ClaireGonzales-NOAA Jul 2, 2025
ba3e383
annual update
ClaireGonzales-NOAA Jul 3, 2025
04eaf61
annual update
ClaireGonzales-NOAA Jul 3, 2025
ac51b94
stratum filtering updates
ClaireGonzales-NOAA Jul 7, 2025
84b7ea7
resolve parsing issue
ClaireGonzales-NOAA Jul 8, 2025
d941f9f
updates to seus
ClaireGonzales-NOAA Jul 9, 2025
48753f3
updates
ClaireGonzales-NOAA Jul 10, 2025
37e509b
some csvs
ClaireGonzales-NOAA Jul 10, 2025
29253be
processed annual update
ClaireGonzales-NOAA Jul 10, 2025
4133fe3
updated table
ClaireGonzales-NOAA Jul 10, 2025
279282f
updating table
ClaireGonzales-NOAA Jul 10, 2025
f218cb3
Updating pathway and annotating
ClaireGonzales-NOAA Jul 10, 2025
874ac95
edit to data read in
ClaireGonzales-NOAA Jul 10, 2025
4b72f3b
updated
ClaireGonzales-NOAA Jul 11, 2025
a468b94
small edit
ClaireGonzales-NOAA Jul 11, 2025
23176d3
updating species key
ClaireGonzales-NOAA Jul 14, 2025
d7a7460
updates to master key
ClaireGonzales-NOAA Jul 15, 2025
40b0fe8
Updated species key
ClaireGonzales-NOAA Jul 16, 2025
c16ecec
Adding core species column for streamlines processing w/ dev team
ClaireGonzales-NOAA Jul 16, 2025
51ebb9b
"Do not use this"
ClaireGonzales-NOAA Jul 16, 2025
3955283
update to filter table annotations
ClaireGonzales-NOAA Jul 16, 2025
81c7e14
remove duplicate csv
ClaireGonzales-NOAA Jul 17, 2025
fb546c8
update filter table
ClaireGonzales-NOAA Jul 17, 2025
018a2dd
updated with annual update
ClaireGonzales-NOAA Jul 17, 2025
0775179
edits to data wrangling
ClaireGonzales-NOAA Jul 17, 2025
1aaee8e
Annual update - added freq of pres in hauls
ClaireGonzales-NOAA Jul 18, 2025
8f364fd
updated csv
ClaireGonzales-NOAA Jul 18, 2025
8c1ae8c
updated wrangling for filter table
ClaireGonzales-NOAA Jul 18, 2025
ebd008b
updated survey name for enbs
ClaireGonzales-NOAA Jul 21, 2025
7b0a2ef
removing unneccessary csv
ClaireGonzales-NOAA Jul 22, 2025
635f8d3
updated code chunks that create tables for the tech report
ClaireGonzales-NOAA Jul 22, 2025
576d781
renaming csvs
ClaireGonzales-NOAA Jul 23, 2025
a1e6554
updating file names
ClaireGonzales-NOAA Jul 23, 2025
89d4c40
species filter
ClaireGonzales-NOAA Aug 5, 2025
0fcaa08
Commented out some csvs
ClaireGonzales-NOAA Aug 5, 2025
05922e0
updated to use df that includes ENBS as a region so that core species…
ClaireGonzales-NOAA Aug 7, 2025
fb200d1
new species filter csv
ClaireGonzales-NOAA Aug 8, 2025
e0a8dcf
Created a script for updating species filter each year
ClaireGonzales-NOAA Aug 8, 2025
f6d7761
update summary table for survey report
ClaireGonzales-NOAA Aug 26, 2025
5ea4fd5
Updated Filter table (`FilterSubRegion`)
ClaireGonzales-NOAA Aug 26, 2025
e90a189
small edit
ClaireGonzales-NOAA Aug 26, 2025
5abb262
update
ClaireGonzales-NOAA Sep 2, 2025
7482b13
update
ClaireGonzales-NOAA Sep 2, 2025
18b892d
update region naming to correctly wrangle Hawaii
ClaireGonzales-NOAA Sep 4, 2025
0ba6447
typo correction
ClaireGonzales-NOAA Nov 14, 2025
bd10f41
reorganizing
ClaireGonzales-NOAA Nov 19, 2025
e1d2978
update to trimming code (lines 2162-2180)
ClaireGonzales-NOAA Nov 19, 2025
938e39c
updating code for species trimming to resolve issues pulling spp/vali…
ClaireGonzales-NOAA Nov 20, 2025
e54af62
Updates to ReadMe that includes the new module
ClaireGonzales-NOAA Dec 1, 2025
140e13c
testing re-analyses
ClaireGonzales-NOAA Dec 3, 2025
347aa35
pull p values for haul freq and biomass regressons
ClaireGonzales-NOAA Dec 8, 2025
bd981fa
updates to trend visualization code
ClaireGonzales-NOAA Dec 8, 2025
b823263
latex fix
ClaireGonzales-NOAA Dec 8, 2025
2ab9f87
testing glm code (shows a warning currently)
ClaireGonzales-NOAA Dec 10, 2025
72a336f
finalize glm() code and annotations
ClaireGonzales-NOAA Dec 16, 2025
0178124
Need to debug data_summary. Error caused when left_join() brings in i…
ClaireGonzales-NOAA Dec 19, 2025
fc2c4b9
debugged
ClaireGonzales-NOAA Dec 19, 2025
16c8328
incorporation of Melissa's feedback for final push
ClaireGonzales-NOAA Dec 30, 2025
319cce0
final push
ClaireGonzales-NOAA Dec 31, 2025
65abe37
code fixes to fix directory paths, and taxonomic matching
Melissa-Karp Jan 14, 2026
4523517
Code review and clean-up
Melissa-Karp Jan 16, 2026
212cf24
Deleted unnecessary files/code clean-up
Melissa-Karp Jan 16, 2026
96046e2
Edits to species filter table, tech document, and compile script
Melissa-Karp Jan 21, 2026
b69ec2f
Merge branch 'nmfs-ost:main' into ncei_dev
john-f-kennedy-noaa Jan 23, 2026
f27fcb1
Merge pull request #19 from john-f-kennedy-noaa/ncei_dev
john-f-kennedy-noaa Jan 23, 2026
6a5e2e0
Merge pull request #21 from nmfs-ost/Python-Processing
john-f-kennedy-noaa Jan 23, 2026
b5fa07b
Resolve merge conflicts
Melissa-Karp Feb 10, 2026
a9c9e34
updated Readme
Melissa-Karp Feb 10, 2026
caeb204
Merge pull request #24 from nmfs-ost/dev-branch
Melissa-Karp Feb 10, 2026
0c6f298
Merge branch 'main' into john-f-kennedy-noaa-metadata
john-f-kennedy-noaa Apr 18, 2026
d4aaf54
metadata
john-f-kennedy-noaa Jun 30, 2026
a57ca21
d
john-f-kennedy-noaa Jun 30, 2026
aa3b528
adding R scripts
john-f-kennedy-noaa Jul 6, 2026
b3fea7c
Perhaps
john-f-kennedy-noaa Jul 6, 2026
87932cd
Merge branch 'john-f-kennedy-noaa-metadata' of https://github.com/joh…
john-f-kennedy-noaa Jul 6, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions - Copy.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#Ignore All
.

#
!/ArcGIS-Analysis-Python
50 changes: 0 additions & 50 deletions .github/copilot-instructions.md

This file was deleted.

52 changes: 4 additions & 48 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,49 +1,5 @@
# History files
.Rhistory
.Rapp.history
.RDataTmp
#Ignore All
.

# Session Data files
.RData

# User-specific files
.Ruserdata

# Example code in package build process
*-Ex.R

# Output files from R CMD build
/*.tar.gz

# Output files from R CMD check
/*.Rcheck/

# RStudio files
.Rproj.user/

# OAuth2 token, see https://github.com/hadley/httr/releases/tag/v0.3
.httr-oauth

# knitr and R markdown default cache directories
*_cache/
/cache/

# Temporary files created by R markdown
*.utf8.md
*.knit.md

# R Environment Variables
.Renviron

/data_processing_rcode/data/
/data_processing_rcode/output/
/data_processing_rcode/notforgit/


/code/.quarto/
*.zip
*.tex
*.log
*.markdown
*.rmarkdown
__init__.py
#
!/ArcGIS-Analysis-Python
23 changes: 23 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# .pre-commit-config.yaml
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.5.0 # Use the latest version
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: check-yaml
- id: check-added-large-files
- id: check-json
- id: detect-private-key
# - repo: https://github.com/psf/black
# rev: 24.3.0 # Use the latest version
# hooks:
# - id: black
- repo: https://github.com/PyCQA/flake8
rev: 7.0.0 # Use the latest version
hooks:
- id: flake8
- repo: https://github.com/PyCQA/isort
rev: 5.13.2 # Use the latest version
hooks:
- id: isort
3 changes: 0 additions & 3 deletions .vscode/settings.json

This file was deleted.

5 changes: 5 additions & 0 deletions ArcGIS-Analysis-Python/ - Copy.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#Ignore All
.

#
!/ArcGIS-Analysis-Python
47 changes: 2 additions & 45 deletions ArcGIS-Analysis-Python/.gitignore
Original file line number Diff line number Diff line change
@@ -1,45 +1,2 @@
# Global ignore: everything
*

# allow directories so we can whitelist specific content
!*/

# Remove selected directories
.ipynb_checkpoints
**/.ipynb_checkpoints
.backups
__pycache__
**/__pycache__


# include python files anywhere
!*.py
!*.pyt
!**/*.py
!**/*.pyt

# still ignore py backup variants
*.~py
**/*.~py
# ignore Python init files globally
__init__.py
**/__init__.py

# include Scripts folder and its contents
!Scripts/
!Scripts/**

# include README files in root, Scripts, and dismap_tools
!README.md
!Scripts/README.md
!src/dismap_tools/README.md

# Notebook folder and its notebooks
Notebook/**
*/secure/**
**/secure/**
user_credentials.py
**/user_credentials.py

# explicitly exclude esri folder under dismap_tools
src/dismap_tools/esri/
#Ignore All
.
23 changes: 23 additions & 0 deletions ArcGIS-Analysis-Python/.pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# .pre-commit-config.yaml
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.5.0 # Use the latest version
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: check-yaml
- id: check-added-large-files
- id: check-json
- id: detect-private-key
# - repo: https://github.com/psf/black
# rev: 24.3.0 # Use the latest version
# hooks:
# - id: black
- repo: https://github.com/PyCQA/flake8
rev: 7.0.0 # Use the latest version
hooks:
- id: flake8
- repo: https://github.com/PyCQA/isort
rev: 5.13.2 # Use the latest version
hooks:
- id: isort
3 changes: 0 additions & 3 deletions ArcGIS-Analysis-Python/August 1 2025/__init__.py

This file was deleted.

6 changes: 0 additions & 6 deletions ArcGIS-Analysis-Python/August 1 2025/user_credentials.py

This file was deleted.

3 changes: 0 additions & 3 deletions ArcGIS-Analysis-Python/Notebooks/__init__.py

This file was deleted.

6 changes: 0 additions & 6 deletions ArcGIS-Analysis-Python/Notebooks/secure/user_credentials.py

This file was deleted.

4 changes: 2 additions & 2 deletions ArcGIS-Analysis-Python/README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# DisMAP ArcGIS Pro Analysis using Python
> This code is always in development. Find the code used for various reports in the code [releases](https://github.com/nmfs-fish-tools/DisMAP/releases).

#### Explanation of this Folder
#### Explanation of this Folder

This folder is part of the overall DisMAP repository and inparticular the ArcGIS Python Process fork. The "src" folder coontains a folder called "dismap_tools", which contains the set of Python scripts to process raw data into other datasets and eventually the interpolated biomass rasters.

Expand All @@ -11,7 +11,7 @@ If you see that the data, product, or metadata can be improved, you are invited

#### NOAA-NMFS GitHub Enterprise Disclaimer

This repository is a scientific product and is not official communication of the National Oceanic and Atmospheric Administration, or the United States Department of Commerce. All NOAA GitHub project code is provided on an ‘as is’ basis and the user assumes responsibility for its use. Any claims against the Department of Commerce or Department of Commerce bureaus stemming from the use of this GitHub project will be governed by all applicable Federal law. Any reference to specific commercial products, processes, or services by service mark, trademark, manufacturer, or otherwise, does not constitute or imply their endorsement, recommendation or favoring by the Department of Commerce.
This repository is a scientific product and is not official communication of the National Oceanic and Atmospheric Administration, or the United States Department of Commerce. All NOAA GitHub project code is provided on an ‘as is’ basis and the user assumes responsibility for its use. Any claims against the Department of Commerce or Department of Commerce bureaus stemming from the use of this GitHub project will be governed by all applicable Federal law. Any reference to specific commercial products, processes, or services by service mark, trademark, manufacturer, or otherwise, does not constitute or imply their endorsement, recommendation or favoring by the Department of Commerce.
The Department of Commerce seal and logo, or the seal and logo of a DOC bureau, shall not be used in any manner to imply endorsement of any commercial product or activity by DOC or the United States Government.

#### NOAA License
Expand Down
23 changes: 0 additions & 23 deletions ArcGIS-Analysis-Python/Scripts/README.md

This file was deleted.

Binary file not shown.
Loading