The Index System provides the foundational architecture for organizing and combining multiple vector spaces into unified searchable structures. Indices serve as the bridge between raw data transformations and executable queries, managing space relationships and schema compatibility.

​
Core Index Components

​
Index Reference

ComponentPurposeSchema IntegrationSpace Management
Index._spacesStores constituent spacesValidates compatibilityInternal space collection
Index._schemasManages schema objectsSchema validation and accessCross-space schema mapping
Index.has_schema()Validates schema compatibilityUsed by QueryDescriptorValidatorBoolean schema checking
Index.__init__()Index constructionCombines spaces with schemasMulti-space initialization

​
Index Creation and Space Combination

An Index combines multiple Space objects into a unified searchable structure through schema compatibility validation and space relationship management. The index maintains references to constituent spaces while ensuring query execution compatibility.

​
Index Components

The Index architecture consists of three primary layers that manage space combination, schema validation, and query integration:
ComponentPurposeImplementation
Index._spacesStores constituent spacesInternal space collection with type checking
Index._schemasManages schema objectsSchema validation and cross-reference mapping
Index.has_schema()Validates schema compatibilityBoolean schema existence checking for queries
The Index manages heterogeneous space collections through internal storage mechanisms:
# Basic index construction
index = sl.Index([
    text_space,
    number_space, 
    recency_space,
    categorical_space
])

# Multi-modal index with image and text
multimedia_index = sl.Index([
    description_space,
    image_space,
    category_space,
    price_space
])
Key Features:
  • Heterogeneous Space Support: Combines different space types in single index
  • Type Validation: Ensures space compatibility during construction
  • Schema Mapping: Maintains relationships between spaces and schema fields
Index construction validates schema compatibility across all constituent spaces:
# Schema-aware index construction
class ProductSchema(sl.Schema):
    description: str
    price: float
    category: str
    image_url: str

index = sl.Index([
    sl.TextSimilaritySpace(ProductSchema.description, model="sentence-transformers/all-MiniLM-L6-v2"),
    sl.NumberSpace(ProductSchema.price, mode=sl.Mode.SIMILAR),
    sl.CategoricalSimilaritySpace(ProductSchema.category),
    sl.ImageSpace(ProductSchema.image_url)
])
Validation Process:
  1. Field Compatibility: Ensures each space references valid schema fields
  2. Type Checking: Validates space-field type compatibility
  3. Cross-Reference Validation: Checks for schema field conflicts
The Index system provides comprehensive metadata management:
# Index with custom metadata
index = sl.Index(
    [text_space, number_space],
    metadata={
        "version": "1.0",
        "description": "Product search index",
        "created": datetime.now()
    }
)

# Schema checking
if index.has_schema(ProductSchema):
    print("Index supports ProductSchema queries")
Metadata Components:
  • Version Tracking: Index version management for compatibility
  • Description Fields: Human-readable index documentation
  • Schema Registry: Available schema types for query validation

​
Index Architecture Patterns

​
Multi-Space Index Construction

​
Index-Schema Relationship Management

The Index system maintains tight integration with schema objects to ensure query compatibility:
Relationship TypePurposeImplementation
Space-Schema BindingLinks spaces to schema fieldsField reference validation
Cross-Space Schema ValidationEnsures schema consistencyMulti-space compatibility checks
Query Schema CompatibilityValidates query target schemasRuntime schema verification
Indices serve as the central coordination point for multi-space vector operations. Proper index construction ensures optimal query performance and maintains data consistency across heterogeneous space types.

​
Advanced Index Features

​
Dynamic Index Configuration

Indices support runtime configuration adjustments for different deployment scenarios: 
# Production index configuration
production_index = sl.Index([
    text_space,
    number_space
], config={
    "optimization_level": "production",
    "cache_strategy": "aggressive",
    "memory_limit": "8GB"
})

# Development index configuration  
dev_index = sl.Index([
    text_space,
    number_space
], config={
    "optimization_level": "development", 
    "cache_strategy": "minimal",
    "debug_mode": True
})

​
Index Composition Patterns

PatternUse CaseBenefits
Homogeneous IndicesSingle space type across fieldsSimplified management, consistent behavior
Heterogeneous IndicesMultiple space types for rich dataComplex queries, multi-modal search
Hierarchical IndicesNested space relationshipsStructured data representation
Federated IndicesDistributed space managementScalability, fault tolerance

​
Usage Patterns

​
Basic Index Creation

# Simple text and number index
basic_index = sl.Index([
    sl.TextSimilaritySpace(schema.description),
    sl.NumberSpace(schema.price, mode=sl.Mode.SIMILAR)
])

​
Complex Multi-Modal Index

# Advanced multi-modal index
complex_index = sl.Index([
    sl.TextSimilaritySpace(schema.title, model="sentence-transformers/all-MiniLM-L6-v2"),
    sl.TextSimilaritySpace(schema.description, model="sentence-transformers/all-mpnet-base-v2"),
    sl.ImageSpace(schema.image_url, model="clip-ViT-B-32"),
    sl.NumberSpace(schema.price, mode=sl.Mode.SIMILAR),
    sl.RecencySpace(schema.created_at, period_time_list=[
        sl.PeriodTime.DAY,
        sl.PeriodTime.WEEK, 
        sl.PeriodTime.MONTH
    ]),
    sl.CategoricalSimilaritySpace(schema.category)
])

​
Index Validation and Compatibility

# Schema compatibility checking
if complex_index.has_schema(ProductSchema):
    # Index supports ProductSchema queries
    query = sl.Query(complex_index).find(ProductSchema)
else:
    raise InvalidInputException("Index does not support ProductSchema")
The Index system forms the foundation of Superlinked’s search architecture, providing the organizational structure needed for complex vector operations while maintaining schema compatibility and type safety across heterogeneous space collections.