KnowledgeForge: Mining and Curating ITSM Ticket Knowledge with AWS

KnowledgeForge is an innovative system designed to extract and organize knowledge from resolved IT Service Management (ITSM) incident tickets. Each ticket contains valuable information—symptoms, root causes, and fixes—that often remains locked within ticket histories, inaccessible to future support engineers. Conversely, traditional knowledge bases tend to become cluttered with duplicates, outdated content, and inconsistent quality.

To address this, KnowledgeForge employs a closed-loop pipeline that mines incident tickets for new knowledge articles and simultaneously curates existing content. This process involves clustering related tickets, generating draft articles using Amazon Bedrock's generative models, and then curating these alongside existing articles through classification, deduplication, quality scoring, and content enhancement. Human oversight is maintained through review and approval in ServiceNow.

The system architecture leverages several AWS services. Amazon Bedrock, with models like Anthropic Claude Sonnet 4.5 and Amazon Titan Text Embeddings V2, powers content generation and similarity detection. Amazon S3 Vectors stores embedding vectors directly in Amazon S3, enabling efficient duplicate detection without a dedicated vector database. AWS Step Functions orchestrate the complex workflow, managing batching, retries, error handling, and parallel processing across multiple compute resources.

Generation begins with clustering related tickets, which are then processed by containers on Amazon ECS with AWS Fargate. These containers retrieve similar existing articles via Amazon S3 Vectors, providing context for the generative models to produce consistent and relevant knowledge articles. The generated content is stored in Amazon S3, where it undergoes curation. This includes classification, duplicate detection, scoring, and content improvement, all orchestrated by AWS Step Functions. Articles passing review are sent to ServiceNow for human approval, with the entire process looping back to improve future generations.

The pipeline incorporates several resilience and efficiency measures. It uses Amazon SQS FIFO queues for ordered processing, dead-letter queues for failed batches, and circuit breakers to prevent runaway failures. Model calls are managed with timeouts, retries, and fallback mechanisms to handle throttling and rate limits. Content improvement employs placeholder tokens to preserve article structure during editing, ensuring high-quality outputs.

A key aspect of KnowledgeForge is its multi-tenant design, isolating customer data through dedicated resources such as vector indexes, prompts, and encryption keys. This ensures data privacy and security.

Lessons learned from internal testing indicate that reusing vector indexes for duplicate detection, passing pointers instead of large content payloads, and scoring content quality before and after improvement are effective strategies for scaling and reliability. The system demonstrates a continuous knowledge improvement cycle, where generated content feeds into curation, which in turn informs subsequent generations.

To implement a similar solution, organizations should start by creating vector indexes, embedding their articles, and experimenting with nearest-neighbor queries for duplicate detection. Layering orchestration workflows and quality gates can further enhance automation and content quality. The complete code and architecture details are available in the aws-samples/sample-knowledgeforge GitHub repository.