Describe Index Stats

Retrieves statistics about a Pinecone index, including total vector count, dimension, and namespace-specific information. This is useful for monitoring index usage and verifying data ingestion.

Common Properties

Name - The custom name of the node.
Color - The custom color of the node.
Delay Before (sec) - Waits in seconds before executing the node.
Delay After (sec) - Waits in seconds after executing node.
Continue On Error - Automation will continue regardless of any error. The default value is false.

info

If the ContinueOnError property is true, no error is caught when the project is executed, even if a Catch node is used.

Inputs

Connection Id - Connection identifier from Connect node (optional if API Key credential is provided directly).
Host URL - Your index's host URL. Format: https://INDEX_NAME-PROJECT_ID.svc.ENVIRONMENT.pinecone.io. You can find this in your Pinecone console.

Options

API Key - Pinecone API key credential (optional - use this instead of Connection Id if not using Connect node).
Filter - Optional metadata filter to get stats for specific vectors only. When provided, returns counts for vectors matching the filter.

Output

Response - Object containing index statistics including dimension, total vector count, and per-namespace breakdown.

{
  "dimension": 1536,
  "indexFullness": 0.4,
  "totalVectorCount": 50000,
  "namespaces": {
    "": {
      "vectorCount": 30000
    },
    "documents": {
      "vectorCount": 15000
    },
    "products": {
      "vectorCount": 5000
    }
  }
}

How It Works

The Describe Index Stats node provides insights into your index's current state. When executed, the node:

Validates required inputs (Host URL)
Constructs the request with optional filter
Sends a POST request to the index's /describe_index_stats endpoint
Returns comprehensive statistics about the index

Requirements

An existing Pinecone index (in any status)
Valid Host URL for the index

Error Handling

The node will return specific errors in the following cases:

ErrInvalidArg - Host URL is empty
ErrInvalidArg - Invalid Connection ID or missing API key
ErrInternal - Response format is not valid
ErrStatus - HTTP error from Pinecone API

Usage Notes

Stats are eventually consistent - recent upserts may take a few seconds to reflect
The empty string key "" in namespaces represents the default namespace
Index fullness ranges from 0 to 1 (0% to 100% capacity)
When filter is provided, stats reflect only vectors matching that filter
This operation does not consume query quota

Understanding the Response

Key Fields

dimension - Vector dimension configured for this index (matches your embedding model)
indexFullness - How full the index is (0-1). Above 0.8 may indicate need for more pods
totalVectorCount - Total number of vectors across all namespaces
namespaces - Breakdown of vector counts per namespace

Index Fullness

The fullness metric helps you plan capacity:

0.0 - 0.5: Plenty of capacity
0.5 - 0.8: Good utilization
0.8 - 1.0: Consider adding pods or cleaning up unused vectors
1.0: Index is full, upserts may fail

Best Practices

Check stats after bulk upserts to verify ingestion
Monitor index fullness to plan capacity upgrades
Use filtered stats to verify specific data segments
Compare namespace counts to ensure proper data distribution
Run this periodically to track growth trends

Example: Monitor Index After Ingestion

// After bulk upsert operation
// Use Describe Index Stats to verify

// Check the response
const stats = msg.resDescribeIndexStats;

console.log(`Total vectors: ${stats.totalVectorCount}`);
console.log(`Index fullness: ${(stats.indexFullness * 100).toFixed(1)}%`);

// Verify expected count
const expectedCount = 10000;
if (stats.totalVectorCount >= expectedCount) {
  console.log("✓ All vectors successfully ingested");
} else {
  console.warn(`⚠ Expected ${expectedCount}, got ${stats.totalVectorCount}`);
}

Example: Check Namespace Distribution

// Get stats
const stats = msg.resDescribeIndexStats;

// Analyze namespace distribution
Object.entries(stats.namespaces).forEach(([namespace, info]) => {
  const name = namespace || "default";
  console.log(`${name}: ${info.vectorCount.toLocaleString()} vectors`);
});

// Calculate percentage by namespace
const total = stats.totalVectorCount;
Object.entries(stats.namespaces).forEach(([namespace, info]) => {
  const percent = ((info.vectorCount / total) * 100).toFixed(1);
  console.log(`${namespace || "default"}: ${percent}%`);
});

Example: Filtered Stats for Data Verification

// Check how many vectors have a specific category
msg.filter = {
  category: "education"
};

// After Describe Index Stats executes with filter
const educationCount = msg.resDescribeIndexStats.totalVectorCount;
console.log(`Education category: ${educationCount} vectors`);

// Compare with total (run again without filter)
msg.filter = null;
// Run Describe Index Stats again
const totalCount = msg.resDescribeIndexStats.totalVectorCount;
const percentage = ((educationCount / totalCount) * 100).toFixed(1);
console.log(`Education is ${percentage}% of total data`);

Example: Capacity Planning

const stats = msg.resDescribeIndexStats;
const fullness = stats.indexFullness;
const totalVectors = stats.totalVectorCount;

// Calculate remaining capacity
const estimatedCapacity = totalVectors / fullness;
const remainingCapacity = estimatedCapacity - totalVectors;

console.log(`Current vectors: ${totalVectors.toLocaleString()}`);
console.log(`Estimated capacity: ${Math.round(estimatedCapacity).toLocaleString()}`);
console.log(`Remaining capacity: ${Math.round(remainingCapacity).toLocaleString()}`);

// Alert if nearing capacity
if (fullness > 0.8) {
  console.warn(`⚠ Index is ${(fullness * 100).toFixed(1)}% full - consider scaling`);
}

Example: Pre-Flight Check Before Query

// Verify index has data before querying
const stats = msg.resDescribeIndexStats;

if (stats.totalVectorCount === 0) {
  throw new Error("Index is empty - no data to query");
}

// Check specific namespace
const namespace = "documents";
if (!stats.namespaces[namespace]) {
  throw new Error(`Namespace '${namespace}' does not exist`);
}

console.log(`✓ Ready to query - ${stats.namespaces[namespace].vectorCount} vectors available`);

Example: Validate Dimension Match

// Ensure your embeddings match index dimension
const stats = msg.resDescribeIndexStats;
const indexDimension = stats.dimension;
const embeddingDimension = msg.myEmbedding.length;

if (indexDimension !== embeddingDimension) {
  throw new Error(
    `Dimension mismatch: Index expects ${indexDimension}, got ${embeddingDimension}`
  );
}

console.log(`✓ Dimension match confirmed: ${indexDimension}`);

Example: Audit Report

const stats = msg.resDescribeIndexStats;

// Generate comprehensive report
const report = {
  timestamp: new Date().toISOString(),
  index: {
    dimension: stats.dimension,
    totalVectors: stats.totalVectorCount,
    fullness: `${(stats.indexFullness * 100).toFixed(1)}%`
  },
  namespaces: Object.entries(stats.namespaces).map(([name, info]) => ({
    name: name || "default",
    vectorCount: info.vectorCount,
    percentage: `${((info.vectorCount / stats.totalVectorCount) * 100).toFixed(1)}%`
  }))
};

console.log(JSON.stringify(report, null, 2));

// Store report for tracking
msg.auditReport = report;

Example: Automated Cleanup Trigger

const stats = msg.resDescribeIndexStats;
const fullness = stats.indexFullness;

// Trigger cleanup if index is getting full
if (fullness > 0.85) {
  console.log("Index nearing capacity - triggering cleanup workflow");

  // Set flag to delete old vectors
  msg.shouldCleanup = true;
  msg.cleanupFilter = {
    last_accessed: {
      "$lt": new Date(Date.now() - 90 * 24 * 60 * 60 * 1000).toISOString() // 90 days
    }
  };
} else {
  msg.shouldCleanup = false;
}

Comparing Before and After Operations

// Before bulk operation
// Run Describe Index Stats
const beforeStats = msg.resDescribeIndexStats;
const beforeCount = beforeStats.totalVectorCount;

// ... perform upsert operations ...

// After bulk operation
// Run Describe Index Stats again
const afterStats = msg.resDescribeIndexStats;
const afterCount = afterStats.totalVectorCount;

const added = afterCount - beforeCount;
console.log(`Added ${added} vectors`);
console.log(`Fullness increased from ${(beforeStats.indexFullness * 100).toFixed(1)}% to ${(afterStats.indexFullness * 100).toFixed(1)}%`);

Troubleshooting

Stats show 0 vectors but data was upserted

Wait a few seconds for eventual consistency
Verify you upserted to the correct index (check Host URL)
Check if data went to a different namespace

Namespace not appearing in stats

Namespace only appears after first successful upsert
Verify namespace name (case-sensitive)
Check for typos in namespace parameter during upsert

Index fullness is 1.0

Index is at capacity - upserts may fail
Add more pods to increase capacity
Delete unused vectors to free space

Dimension doesn't match my embeddings

You're using the wrong index
Your embedding model changed
Check your embedding generation code

Filtered stats return unexpected counts

Filter syntax may be incorrect
Not all vectors have the metadata field you're filtering on
Metadata values may not match exactly (check types and case)

Common Properties​

Inputs​

Options​

Output​

How It Works​

Requirements​

Error Handling​

Usage Notes​

Understanding the Response​

Key Fields​

Index Fullness​

Best Practices​

Example: Monitor Index After Ingestion​

Example: Check Namespace Distribution​

Example: Filtered Stats for Data Verification​

Example: Capacity Planning​

Example: Pre-Flight Check Before Query​

Example: Validate Dimension Match​

Example: Audit Report​

Example: Automated Cleanup Trigger​

Comparing Before and After Operations​

Troubleshooting​