The Knowledge Graph Construction Pipeline
Building a knowledge graph from raw data involves several key stages. The process takes unstructured or semi-structured data and transforms it into a connected, queryable graph.
Pipeline
Raw Data
CSV files, JSON APIs, databases, text documents — your source data in any format.
Raw Datapython
import pandas as pd
# Load raw data from CSV
df = pd.read_csv("employees.csv")
print(df.head())
# name, age, company, city
# Alice, 32, Acme Corp, London
# Bob, 28, Acme Corp, London
# Carol, 35, Globex Inc, New YorkBuilding with Python rdflib
Python's rdflib library is the standard tool for working with RDF data programmatically. Here's a complete example:
Complete knowledge graph construction with rdflibpython
from rdflib import Graph, Namespace, URIRef, Literal, BNode
from rdflib.namespace import FOAF, RDF, RDFS, XSD, OWL
import pandas as pd
# Define namespaces
EX = Namespace("http://example.org/")
ORG = Namespace("http://www.w3.org/ns/org#")
SCHEMA = Namespace("https://schema.org/")
g = Graph()
g.bind("ex", EX)
g.bind("foaf", FOAF)
g.bind("org", ORG)
g.bind("schema", SCHEMA)
# ── 1. Define schema (T-Box) ──────────────────────────
g.add((EX.Employee, RDF.type, OWL.Class))
g.add((EX.Employee, RDFS.subClassOf, FOAF.Person))
g.add((EX.Employee, RDFS.label, Literal("Employee")))
g.add((EX.worksAt, RDF.type, OWL.ObjectProperty))
g.add((EX.worksAt, RDFS.domain, EX.Employee))
g.add((EX.worksAt, RDFS.range, ORG.Organization))
# ── 2. Load raw data ──────────────────────────────────
df = pd.read_csv("employees.csv")
company_cache: dict[str, URIRef] = {}
city_cache: dict[str, URIRef] = {}
for _, row in df.iterrows():
# Person URI
slug = row["name"].lower().replace(" ", "_")
person = EX[f"person/{slug}"]
g.add((person, RDF.type, EX.Employee))
g.add((person, FOAF.name, Literal(row["name"])))
g.add((person, FOAF.age, Literal(int(row["age"]), datatype=XSD.integer)))
# Company URI (deduplication via cache)
company_name = row["company"]
if company_name not in company_cache:
co_slug = company_name.lower().replace(" ", "_")
company = EX[f"company/{co_slug}"]
company_cache[company_name] = company
g.add((company, RDF.type, ORG.Organization))
g.add((company, FOAF.name, Literal(company_name)))
company = company_cache[company_name]
# City URI
city_name = row["city"]
if city_name not in city_cache:
city_slug = city_name.lower().replace(" ", "_")
city = EX[f"city/{city_slug}"]
city_cache[city_name] = city
g.add((city, RDF.type, SCHEMA.City))
g.add((city, FOAF.name, Literal(city_name)))
city = city_cache[city_name]
# Relationships
g.add((person, EX.worksAt, company))
g.add((company, SCHEMA.location, city))
print(f"Graph has {len(g)} triples")
# ── 3. Serialize ──────────────────────────────────────
g.serialize(destination="knowledge_graph.ttl", format="turtle")
g.serialize(destination="knowledge_graph.jsonld", format="json-ld")
# ── 4. Query ──────────────────────────────────────────
result = g.query("""
SELECT ?name ?company WHERE {
?p foaf:name ?name ;
ex:worksAt ?c .
?c foaf:name ?company .
}
""", initNs={"foaf": FOAF, "ex": EX})
for row in result:
print(f"{row.name} works at {row.company}")Entity linking to Wikidata using SPARQLWrapperpython
from SPARQLWrapper import SPARQLWrapper, JSON
def link_city_to_wikidata(city_name: str) -> str | None:
"""Look up a city in Wikidata and return its QID."""
sparql = SPARQLWrapper("https://query.wikidata.org/sparql")
sparql.setQuery(f"""
SELECT ?item WHERE {{
?item wdt:P31 wd:Q515 ; # instance of: city
rdfs:label "{city_name}"@en .
}}
LIMIT 1
""")
sparql.setReturnFormat(JSON)
results = sparql.query().convert()
if results["results"]["bindings"]:
uri = results["results"]["bindings"][0]["item"]["value"]
return uri # e.g. "https://www.wikidata.org/entity/Q84"
return None
# Add owl:sameAs link to Wikidata
from rdflib import OWL, URIRef
wikidata_uri = link_city_to_wikidata("London")
if wikidata_uri:
g.add((EX["city/london"], OWL.sameAs, URIRef(wikidata_uri)))
print(f"Linked London → {wikidata_uri}")Graph Stores: Where to Persist Your Graph
Once you've built your graph, you need a triple store to persist and query it at scale:
| Store | Type | Best for |
|---|---|---|
| Apache Jena Fuseki | Open-source, Java | Development, research |
| Stardog | Commercial | Enterprise, rules, ML |
| GraphDB | Commercial/Free tier | Semantic reasoning, compliance |
| Amazon Neptune | Managed cloud | AWS environments |
| Blazegraph | Open-source | Wikidata, large-scale |
| Virtuoso | Open-source/Commercial | DBpedia, linked data |
Summary
In this chapter you learned:
- The four-stage KG construction pipeline: Raw Data → Entity Extraction → Triple Generation → Graph Store
- rdflib in Python lets you programmatically build RDF graphs
- Entity linking connects local entities to global knowledge bases like Wikidata
- Triple stores persist and serve your graph via SPARQL endpoints
Next, we'll explore how knowledge graphs can automatically generate new facts through inference and reasoning.