Information Extraction & NER — AI Tutorials

Information Extraction

Information Extraction (IE) transforms unstructured text into structured data — converting natural language into facts, relations, and events that can be stored in databases and queried.

The IE Hierarchy

Text: "Apple CEO Tim Cook announced a $90B buyback in Cupertino on Tuesday."
  │
  ├─ NER: [Apple]ORG [Tim Cook]PERSON [$90B]MONEY [Cupertino]GPE [Tuesday]DATE
  │
  ├─ Relation: (Tim Cook, CEO_OF, Apple), (Apple, HEADQUARTERED_IN, Cupertino)
  │
  └─ Event: {type: Announcement, agent: Tim Cook, org: Apple,
             amount: $90B, location: Cupertino, date: Tuesday}

Named Entity Recognition (NER)

NER identifies and classifies named entities in text. Standard entity types (ACE 2005 / OntoNotes):

Tag	Entity Type	Example
PERSON	People	"Marie Curie", "Tim Cook"
ORG	Organizations	"Google", "UN", "Harvard"
GPE	Geo-Political Entities	"France", "New York City"
LOC	Non-GPE Locations	"the Amazon River", "the Alps"
DATE	Dates and time periods	"Tuesday", "Q3 2024"
TIME	Specific times	"3:45 PM", "noon"
MONEY	Monetary values	"$90 billion", "€500"
PERCENT	Percentages	"15%", "three quarters"
PRODUCT	Products/works	"iPhone 16", "Harry Potter"
EVENT	Named events	"World War II", "the Olympics"

NER Approaches

IOB/BIOES Tagging

"Tim Cook visited New York City on Tuesday ."
  B-PER I-PER    O       B-GPE I-GPE I-GPE O  B-DATE  O

BIOES variant:
  B-PER E-PER    O       B-GPE I-GPE E-GPE O  S-DATE  O

Rule-Based NER

Early systems used hand-crafted rules and gazetteers (entity dictionaries):

Capitalized word after sentence start → PERSON candidate
Word in a country list → GPE
"$" followed by digits → MONEY

Fast and precise for known entity lists but brittle for novel entities.

CRF-Based NER

Conditional Random Fields model the joint probability of the entire tag sequence:

$P(y | x) = \frac{1}{Z(x)} \exp\left(\sum_{t=1}^{T} \sum_k \lambda_k f_k(y_t, y_{t-1}, x, t)\right)$

Features include word identity, prefix/suffix, capitalization, surrounding words, and part-of-speech tags. CRFs enforce valid tag transitions (B-PER must be followed by I-PER or a non-PER tag, not I-ORG).

BERT-Based NER

State-of-the-art NER fine-tunes BERT with a token classification head:

Input:  [CLS] Tim  Cook  visited New   York  [SEP]
BERT:   h_cls h_Tim h_Cook h_vis h_New h_York h_sep
Linear: —     B-PER I-PER  O     B-GPE I-GPE  —

Performance (CoNLL-2003 English F1):

Rule-based: ~70–75%
CRF with hand features: ~88–90%
BiLSTM-CRF: ~90–91%
BERT-large fine-tuned: ~92–93%
RoBERTa / DeBERTa: ~93–94%

Nested NER

Standard IOB cannot handle overlapping entities:

"Bank of America headquarters" → [Bank of America]ORG, [America]GPE

Solutions: span-based models that score all possible spans independently, or hierarchical tagging.

Relation Extraction

Relation Extraction (RE) identifies semantic relationships between entity pairs:

$\text{(Tim Cook, CEO\_OF, Apple)} \quad \text{(Apple, HEADQUARTERED\_IN, Cupertino)}$

Pipeline vs Joint Approaches

Approach	Pipeline	Joint
Process	NER → RE separately	NER + RE together
Pros	Modular, easy to debug	No error propagation
Cons	Errors compound	More complex training

RE Methods

1. Pattern-based: "X, CEO of Y" → (X, CEO_OF, Y). High precision, low recall.

2. Supervised: Classify relation type for (entity1, entity2) pairs. Features include entity types, words between entities, and dependency path.

3. BERT-based: Fine-tune on (sentence, entity1, entity2) → relation type. Special entity marker tokens: "[[Tim Cook]] is the CEO of ((Apple))."

4. OpenIE (Open Information Extraction): Extracts (subject, relation, object) triples without predefined relation types:

"Einstein developed the theory of relativity" → (Einstein, developed, theory of relativity)

NER with spaCy, HuggingFace, and Relation Extractionpython

# ─── 1. spaCy NER ────────────────────────────────────────────
import spacy
from spacy import displacy

nlp = spacy.load("en_core_web_sm")

text = """Apple CEO Tim Cook announced a $90 billion stock buyback program
at the company's headquarters in Cupertino, California on Tuesday.
The announcement came after Apple reported record quarterly revenue of
$94.9 billion, a 5% increase from last year."""

doc = nlp(text)
print("Named Entities:")
print(f"{'Entity':<30} {'Label':<12} {'Start':>6} {'End':>6}")
print("-" * 58)
for ent in doc.ents:
    print(f"{ent.text:<30} {ent.label_:<12} {ent.start_char:>6} {ent.end_char:>6}")

# Entity grouping by type
from collections import defaultdict
entity_groups = defaultdict(list)
for ent in doc.ents:
    entity_groups[ent.label_].append(ent.text)

print("\nEntities by type:")
for label, entities in entity_groups.items():
    print(f"  {label}: {list(set(entities))}")

# ─── 2. BERT NER (HuggingFace) ───────────────────────────────
from transformers import pipeline

# Pre-trained NER model (CoNLL-2003 fine-tuned)
ner = pipeline("ner",
               model="dbmdz/bert-large-cased-finetuned-conll03-english",
               aggregation_strategy="simple")  # merges B/I tokens

results = ner(text)
print("\nBERT NER Results:")
for ent in results:
    print(f"  [{ent['entity_group']:<8}] {ent['word']:<25} score={ent['score']:.3f}")

# ─── 3. Custom NER with spaCy Training ───────────────────────
import spacy
from spacy.training import Example

# Create blank model and add NER component
nlp_custom = spacy.blank("en")
ner_pipe = nlp_custom.add_pipe("ner")

# Add entity labels
ner_pipe.add_label("DRUG")
ner_pipe.add_label("DISEASE")
ner_pipe.add_label("SYMPTOM")

# Training data format
train_data = [
    ("Aspirin reduces fever and headaches.",
     {"entities": [(0, 7, "DRUG"), (16, 21, "SYMPTOM"), (26, 35, "SYMPTOM")]}),
    ("Metformin is used to treat type 2 diabetes.",
     {"entities": [(0, 9, "DRUG"), (27, 43, "DISEASE")]}),
]

# Convert to spaCy Example format
examples = []
for text, annotations in train_data:
    doc = nlp_custom.make_doc(text)
    example = Example.from_dict(doc, annotations)
    examples.append(example)

# Initialize and train (simplified — real training needs more data/epochs)
nlp_custom.initialize(lambda: examples)
# for epoch in range(10):
#     nlp_custom.update(examples, drop=0.3)

# ─── 4. Relation Extraction ──────────────────────────────────
# Using OpenIE-style extraction with spaCy
def extract_svo_triples(doc):
    """Extract Subject-Verb-Object triples."""
    triples = []
    for token in doc:
        if token.dep_ == "ROOT" and token.pos_ == "VERB":
            subjects = [w for w in token.lefts if w.dep_ in ("nsubj", "nsubjpass")]
            objects = [w for w in token.rights if w.dep_ in ("dobj", "pobj", "attr")]
            for subj in subjects:
                for obj in objects:
                    # Get full noun phrases
                    subj_phrase = " ".join([t.text for t in subj.subtree
                                           if t.dep_ not in ("punct",)])
                    obj_phrase = " ".join([t.text for t in obj.subtree
                                          if t.dep_ not in ("punct",)])
                    triples.append((subj_phrase, token.lemma_, obj_phrase))
    return triples

sentences = [
    "Apple acquired Beats Electronics for $3 billion in 2014.",
    "Tim Cook leads Apple as its chief executive officer.",
    "Google developed the TensorFlow machine learning framework.",
]
print("\nSVO Relation Triples:")
for sent in sentences:
    doc = nlp(sent)
    triples = extract_svo_triples(doc)
    for subj, verb, obj in triples:
        print(f"  ({subj!r}, {verb!r}, {obj!r})")

# ─── 5. Entity Linking (NEL) ─────────────────────────────────
# Link entities to Wikidata/Wikipedia
# pip install spacy-entity-linker
# python -m spacy_entity_linker "download_knowledge_base"

# Example with spacy-transformers EntityLinker
# nlp = spacy.load("en_core_web_trf")
# nlp.add_pipe("entityLinker", last=True)
# doc = nlp("Marie Curie won the Nobel Prize in Physics.")
# for ent in doc.ents:
#     print(f"{ent.text} → {ent._.linkedEntities}")

Event Extraction

Event extraction identifies structured events: what happened, who did it, where, when.

ACE Event Schema

The ACE (Automatic Content Extraction) annotation defines:

Event trigger: the word/phrase that signals the event (e.g., "acquired", "died", "attacked")
Event type: one of 33 types (e.g., BUSINESS.Merge-Org, LIFE.Die, CONFLICT.Attack)
Event arguments: roles filled by entities (Agent, Patient, Target, Time, Place, ...)

"Apple acquired Beats for $3 billion in 2014."
   Trigger: "acquired" → BUSINESS.Merge-Org
   Arg-Org: "Apple" (Buyer)
   Arg-Org: "Beats" (Artifact/Seller)
   Arg-Money: "$3 billion" (Price)
   Arg-Time: "2014" (Time)

Knowledge Base Population

IE feeds knowledge bases used for:

Question answering: Wikidata, Freebase, DBpedia
Recommendation: entity graphs connecting movies, actors, directors
Search: Google Knowledge Graph, entity cards
RAG pipelines: structured retrieval alongside dense embeddings

Coreference in IE

Before extraction, coreference resolution (Chapter 6) is essential:

"Apple CEO Tim Cook... He announced..." → resolve "He" → "Tim Cook"
Without resolution, the extracted facts may be incomplete or incorrect.

Knowledge check

In the IOB tagging scheme, which tag sequence correctly labels "New York City" as a GPE entity?

Summary

NER identifies entity spans and types; IOB/BIOES tagging is the standard encoding for sequence labeling
BERT-based NER achieves ~93% F1 on CoNLL-2003; domain-specific models (biomedical, legal) require specialized fine-tuning
Relation Extraction extracts (subject, relation, object) triples; joint NER+RE models avoid error propagation
Event Extraction captures structured events with triggers, types, and argument roles (ACE schema)
Knowledge Base Population uses IE to build and populate knowledge graphs for QA, search, and RAG
Entity Linking connects extracted entities to canonical knowledge base entries (Wikidata, Wikipedia)

Next: Machine Translation — sequence-to-sequence models and neural translation systems.

← Natural Language Processing