FatimaInstitute/3d-genomics-prediction
22.8 GB
AI & ML interests
None defined yet.
Recent Activity
View all activity
sa501428Â
updated a
bucket 1 day ago
sa501428Â
published a
bucket 1 day ago
sa501428Â
published a
dataset 9 days ago
Post
1148
Uhh did Opus 4.8 cheat on PostTrainBench??
it found an API key in the PostTrainBench environment that allowed it to generate synthetic training data without using GPU hours, boosting the base model by 0.4913
Source: https://posttrainbench.com/traces/run.html?id=claude_non_api_max_claude-opus-4-8_10h_run1__healthbench_Qwen_Qwen3-4B-Base_17315102#tab=trace
it found an API key in the PostTrainBench environment that allowed it to generate synthetic training data without using GPU hours, boosting the base model by 0.4913
Source: https://posttrainbench.com/traces/run.html?id=claude_non_api_max_claude-opus-4-8_10h_run1__healthbench_Qwen_Qwen3-4B-Base_17315102#tab=trace
- 1 reply
Sajib-006Â
authored a
paper 2 months ago
Post
186
Excited to share our paper: Think Again or Think Longer? Selective Verification for Budget-Aware Reasoning
A common assumption in test-time reasoning is that giving a model more chances to think or verify should improve performance. Our results show that this is only partly true.
We introduce SEVRA, a serving-layer controller that decides when a frozen reasoning model should keep its initial answer and when it should actively verify it. Instead of treating verification as always useful, SEVRA asks a more deployment-focused question:
Is this specific attempt likely recoverable by verification?
We evaluate this through helpful fixes, harmful flips, extra calls, and realized token cost.
Some key takeaways:
* Selective verification improves over always verifying on MATH500 while reducing harmful flips.
* On GSM8K, the controller verifies only a small fraction of examples but still improves accuracy.
* However, a longer initial solve can sometimes match selective verification with fewer realized tokens.
* Cheap serving-visible features, such as completion status, token count, and finalizer use, nearly match larger learned gates.
* On CommonsenseQA, always-on verification hurts, showing that the best test-time compute action is workload-dependent.
The main deployment lesson is simple:
Tune the initial reasoning budget first. Then use selective recovery when explicit checks, bounded retries, auditability, or regression-risk control matter.
Paper: Think Again or Think Longer? Selective Verification for Budget-Aware Reasoning (2606.19808)
Code: https://github.com/Sajib-006/SEVRA
Replay dashboard: sevra-space/sevra-replay
Would love feedback from the community, especially on broader test-time compute allocation, risk-aware verification, and practical serving policies for reasoning models.
A common assumption in test-time reasoning is that giving a model more chances to think or verify should improve performance. Our results show that this is only partly true.
We introduce SEVRA, a serving-layer controller that decides when a frozen reasoning model should keep its initial answer and when it should actively verify it. Instead of treating verification as always useful, SEVRA asks a more deployment-focused question:
Is this specific attempt likely recoverable by verification?
We evaluate this through helpful fixes, harmful flips, extra calls, and realized token cost.
Some key takeaways:
* Selective verification improves over always verifying on MATH500 while reducing harmful flips.
* On GSM8K, the controller verifies only a small fraction of examples but still improves accuracy.
* However, a longer initial solve can sometimes match selective verification with fewer realized tokens.
* Cheap serving-visible features, such as completion status, token count, and finalizer use, nearly match larger learned gates.
* On CommonsenseQA, always-on verification hurts, showing that the best test-time compute action is workload-dependent.
The main deployment lesson is simple:
Tune the initial reasoning budget first. Then use selective recovery when explicit checks, bounded retries, auditability, or regression-risk control matter.
Paper: Think Again or Think Longer? Selective Verification for Budget-Aware Reasoning (2606.19808)
Code: https://github.com/Sajib-006/SEVRA
Replay dashboard: sevra-space/sevra-replay
Would love feedback from the community, especially on broader test-time compute allocation, risk-aware verification, and practical serving policies for reasoning models.
Sajib-006Â
submitted a
paper to Daily Papers 3 months ago
Sajib-006Â
authored 3
papers 3 months ago
LLM4Cell: A Survey of Large Language and Agentic Models for Single-Cell Biology
Paper • 2510.07793 • Published • 3
PlantMarkerBench: A Multi-Species Benchmark for Evidence-Grounded Plant Marker Reasoning
Paper • 2605.10032 • Published • 3
BenSyc: Benchmarking Conversational Sycophancy and Human Alignment in LLMs for Bengali Contexts
Paper • 2606.10061 • Published
Sajib-006Â
submitted a
paper to Daily Papers 3 months ago
Post
200
Excited to share BenSyc v1.1 — a Bengali conversational sycophancy benchmark with privacy-redacted binary and five-class releases. Explore the full dataset and project Space here:
Dataset: Sajib-006/bensyc
Space: Sajib-006/bensyc-project
#BengaliNLP #DatasetRelease #AIAlignment
Dataset: Sajib-006/bensyc
Space: Sajib-006/bensyc-project
#BengaliNLP #DatasetRelease #AIAlignment
amayuelasÂ
authored a
paper 4 months ago
Sajib-006Â
submitted a
paper to Daily Papers 4 months ago
safikhanÂ
authored 6
papers 4 months ago
MILU: A Multi-task Indic Language Understanding Benchmark
Paper • 2411.02538 • Published • 3
BhasaAnuvaad: A Speech Translation Dataset for 14 Indian Languages
Paper • 2411.04699 • Published • 1
Pralekha: An Indic Document Alignment Evaluation Benchmark
Paper • 2411.19096 • Published
Can Vision-Language Models Evaluate Handwritten Math?
Paper • 2501.07244 • Published
FairI Tales: Evaluation of Fairness in Indian Contexts with a Focus on Bias and Stereotypes
Paper • 2506.23111 • Published
IndicIFEval: A Benchmark for Verifiable Instruction-Following Evaluation in 14 Indic Languages
Paper • 2602.22125 • Published