Skip to content

Downgrade OSSF scorecard to 2.4.3 and bump ntlk #170

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mariusarvinte merged 2 commits into from
Oct 9, 2026
+17 −14
Merged

Downgrade OSSF scorecard to 2.4.3 and bump ntlk #170

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
8d56874
Downgrade OSSF scorecard
mariusarvinte Oct 9, 2026
5588a2c
Drop ntlk as permanent dependency
mariusarvinte Oct 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .github/workflows/scorecard.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,8 @@ jobs:
ref: main

- name: "Run analysis"
uses: ossf/scorecard-action@2d1146689b8cda280b9bc96326124645441f03bc # v2.4.4
# v2.4.4 hangs on requirements.txt resolution (https://github.com/ossf/scorecard/issues/5234)
uses: ossf/scorecard-action@4eaacf0543bb3f2c246792bd56e8cdeffafb205a # v2.4.3
with:
results_file: results.sarif
results_format: sarif
Expand Down
16 changes: 12 additions & 4 deletions examples/fact_checking/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,19 @@ include ../../preamble.mk

num_steps=2
args=--num_steps $(num_steps)
nltk=nltk==3.10.3

all: run

run:
> $(RUN_GPU) --with-requirements requirements.txt claim.py $(args)
> $(RUN_GPU) --with-requirements requirements.txt document.py $(args)
nltk_data:
> uv run --isolated --no-project --with $(nltk) -m nltk.downloader punkt_tab

.PHONY: all run
claim: nltk_data
> $(RUN_GPU) --with-requirements requirements.txt --with $(nltk) claim.py $(args)

document: nltk_data
> $(RUN_GPU) --with-requirements requirements.txt --with $(nltk) document.py $(args)

run: claim document

.PHONY: all nltk_data claim document run
11 changes: 3 additions & 8 deletions examples/fact_checking/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,22 +1,17 @@
# Basics and requirements
Install `uv` and `llmart`, and download/navigate to this folder.

If this is the first time running this example on a machine and an error related to `ntlk` occurs, it can be resolved by manually running:
```bash
uv run --with nltk==3.9.1 -m nltk.downloader "punkt_tab"
```

## White-box attacks with `llmart`

The attacks run end-to-end adversarial optimization on the fact-checking task used by the MiniCheck framework.

MiniCheck paper: https://arxiv.org/abs/2404.10774 \
MiniCheck repository: https://github.com/Liyan06/MiniCheck

Given a claim and a document, appending adversarial suffixes for either can be run using the commands:
Given a claim and a document, appending adversarial suffixes for either can be run using the command below, which also downloads the required NLTK `punkt_tab` tokenizer data:
```bash
uv run --with-requirements requirements.txt document.py
uv run --with-requirements requirements.txt claim.py
make document num_steps=1000
make claim num_steps=1000
```

> [!NOTE]
Expand Down
1 change: 0 additions & 1 deletion examples/fact_checking/requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,2 +1 @@
fire==0.7.0
nltk==3.9.1
Loading