| --- |
| license: mit |
| language: |
| - en |
| tags: |
| - statement-extraction |
| - named-entity-recognition |
| - t5 |
| - gemma |
| - seq2seq |
| - nlp |
| - information-extraction |
| - corp-o-rate |
| pipeline_tag: text2text-generation |
| --- |
| |
| # Statement Extractor (T5-Gemma 2) |
|
|
| A fine-tuned T5-Gemma 2 model for extracting structured statements from text. Part of [corp-o-rate.com](https://corp-o-rate.com). |
|
|
| ## Model Description |
|
|
| This model extracts subject-predicate-object triples from unstructured text, with automatic entity type recognition and coreference resolution. |
|
|
| - **Architecture**: T5-Gemma 2 (270M-270M, 540M total parameters) |
| - **Training Data**: 77,515 examples from corporate and news documents |
| - **Final Eval Loss**: 0.209 |
| - **Max Input Length**: 4,096 tokens |
| - **Max Output Length**: 2,048 tokens |
|
|
| ## Usage |
|
|
| ### Python |
|
|
| ```python |
| from transformers import AutoModelForSeq2SeqLM, AutoTokenizer |
| import torch |
| |
| model = AutoModelForSeq2SeqLM.from_pretrained( |
| "Corp-o-Rate-Community/statement-extractor", |
| torch_dtype=torch.bfloat16, |
| trust_remote_code=True, |
| ) |
| tokenizer = AutoTokenizer.from_pretrained( |
| "Corp-o-Rate-Community/statement-extractor", |
| trust_remote_code=True, |
| ) |
| |
| text = "Apple Inc. announced a commitment to carbon neutrality by 2030." |
| inputs = tokenizer(f"<page>{text}</page>", return_tensors="pt") |
| outputs = model.generate(**inputs, max_new_tokens=2048, num_beams=4) |
| result = tokenizer.decode(outputs[0], skip_special_tokens=True) |
| print(result) |
| ``` |
|
|
| ### Input Format |
|
|
| Wrap your text in `<page>` tags: |
|
|
| ``` |
| <page>Your text here...</page> |
| ``` |
|
|
| ### Output Format |
|
|
| The model outputs XML with extracted statements: |
|
|
| ```xml |
| <statements> |
| <stmt> |
| <subject type="ORG">Apple Inc.</subject> |
| <object type="EVENT">carbon neutrality by 2030</object> |
| <predicate>committed to</predicate> |
| <text>Apple Inc. committed to achieving carbon neutrality by 2030.</text> |
| </stmt> |
| </statements> |
| ``` |
|
|
| ## Entity Types |
|
|
| | Type | Description | |
| |------|-------------| |
| | ORG | Organizations (companies, agencies) | |
| | PERSON | People (names, titles) | |
| | GPE | Geopolitical entities (countries, cities) | |
| | LOC | Locations (mountains, rivers) | |
| | PRODUCT | Products (devices, services) | |
| | EVENT | Events (announcements, meetings) | |
| | WORK_OF_ART | Creative works (reports, books) | |
| | LAW | Legal documents | |
| | DATE | Dates and time periods | |
| | MONEY | Monetary values | |
| | PERCENT | Percentages | |
| | QUANTITY | Quantities and measurements | |
|
|
| ## Demo |
|
|
| Try the interactive demo at [statement-extractor.vercel.app](https://statement-extractor.vercel.app) |
|
|
| ## Training |
|
|
| - Base model: `google/t5gemma-2-270m-270m` |
| - Training examples: 77,515 |
| - Final eval loss: 0.209 |
| - Training with refinement phase (LR=1e-6, epochs=0.2) |
| - Beam search: num_beams=4 |
| |
| ## About corp-o-rate |
| |
| This model is part of [corp-o-rate.com](https://corp-o-rate.com) - an AI-powered platform for ESG analysis and corporate accountability. |
| |
| ## License |
| |
| MIT License |
| |
