A Proposal from DeepSeek
At Actualog PIM, we’re on an eternal quest to tame the chaos of product data. We meticulously craft and continuously refine product templates, also known as Item Identification Guides (IIGs) — structured lists of attributes for each product category. These templates align with ISO 22745, the international standard that ensures product data doesn’t turn into an unmanageable digital landfill.
To process semi-structured product data, we need a smart solution that can:
✅ Decipher if a record represents an actual product
✅ Identify the correct product category
✅ Extract relevant attribute values from product descriptions, invoices, and other sources.
So, naturally, we turned to AI. We asked our AI Team for guidance—specifically DeepSeek, because ChatGPT was busy coding in .NET 9 and didn’t pick up the phone.
Here’s what DeepSeek proposed
The core idea is to leverage modern AI models to process unstructured product data, determine whether a record represents a valid product, assign it to the correct category, and extract structured attributes using both text analysis and schema mapping.
The proposed workflow consists of five key stages:
1️⃣ Data Ingestion & Preprocessing
2️⃣ Product Category Classification
3️⃣ Attribute Extraction & Validation
4️⃣ Integration with .NET 9 & Azure AI
5️⃣ Deployment & Monitoring
1. Data Ingestion & Preprocessing
The first challenge is handling unstructured product data from various sources (PDFs, images, Office files).
Proposed Solution:
- Use Azure AI Document Intelligence to convert unstructured files into structured Markdown or JSON.
- Extract layout, text, and hierarchical information to simplify downstream AI processing.
🔹 Example Use Case:
Extract product descriptions, invoices, or contracts while preserving structure (e.g., headings, tables) for better AI interpretation.
AI-Based Text Understanding
- Generate vector embeddings from product descriptions using Azure OpenAI’s
text-embedding-ada-002. - Store embeddings in a vector database (Qdrant or Azure Cognitive Search) to enable semantic search for similar products.
📝 Key Question: How well does Azure AI Document Intelligence handle complex product descriptions, and what preprocessing might be needed?
2. AI-Driven Product Category Classification
After preprocessing, the AI needs to determine the correct product category.
Proposed AI Classification Methods:
Zero-Shot Learning with GPT-4
Use GPT-4/4o to classify products without training data by prompting it with predefined category lists.
🔹 Example AI Prompt:
<code>var prompt = $"Classify this product into one of [{categories}]. Product: {description}";<br>var completion = await chatClient.CompleteChatAsync([new UserChatMessage(prompt)]); <br></code>📝 Key Question: Does GPT-4’s zero-shot learning provide sufficient accuracy, or would a fine-tuned model be needed?
Hybrid Approach: Combining Embeddings & ML.NET
- If labeled data is available, ML.NET could be used alongside embeddings to improve classification accuracy.
- Storing embeddings in a vector database would enable fast similarity-based classification.
📝 Key Question: Would a supervised learning approach be more effective than a purely zero-shot method?
Dynamic Schema Matching for Attributes
- Leverage Azure OpenAI’s function-calling to match extracted product data to Item Identification Guides (IIGs) based on ISO 22745.
🔹 Example AI Prompt:
"Extract attributes matching this schema: {schema}. Product details: {text}"📝 Key Question: How do we ensure AI-generated attributes align with industry-standard schemas?
3. Attribute Extraction & Validation
Once a product is categorized, we need to extract and validate its attributes.
Proposed Attribute Extraction Method:
- Use GPT-4 Turbo in JSON mode to transform unstructured text into structured attribute-value pairs (e.g., “Color: Blue, Screen Size: 6.7 inches”).
- Integrate with System.Text.Json in .NET 9 for efficient serialization.
Data Validation Using AI
- Use Semantic Kernel to validate attributes against PIM rules (e.g., SKU format, missing attributes).
- Detect outliers or inconsistencies before storing data in the PIM database.
📝 Key Question: How do we handle edge cases where AI extraction might misinterpret values (e.g., “6.7 inches” being confused with a weight measurement)?
4. Integration with .NET 9 & Azure AI
Proposed AI & Backend Integration:
- Use Microsoft.Extensions.AI and Microsoft.Extensions.VectorData in .NET 9 for seamless interaction with AI models and vector databases.
🔹 Example: Adding AI Services in .NET 9
<code>services.AddAzureOpenAIChatClient("gpt-4", endpoint, credentials);<br>services.AddQdrantVectorDatabase("qdrant-url"); <br></code>Performance Optimization Considerations
- Use Native AOT compilation in .NET 9 to improve startup time for AI-driven microservices.
- Implement HybridCache to cache frequently accessed category embeddings.
📝 Key Question: What trade-offs exist between real-time AI processing vs. caching pre-computed results?
5. Cloud-Native Deployment & Monitoring
Proposed Deployment Strategy:
- Deploy the solution using Azure Container Apps or AKS.
- Use .NET Aspire for orchestration and resource management.
AI Model Monitoring & Continuous Improvement
- Use Azure Monitor & Application Insights to track:
- Classification accuracy rates
- Processing latency
- Misclassification patterns
📝 Key Question: How frequently should AI models be retrained or fine-tuned based on misclassified data?
Proposed Workflow Overview
1️⃣ User Uploads Document → AI extracts structured content from a PDF, invoice, or catalog.
2️⃣ AI Classifies Product → GPT-4 assigns a category (e.g., “Electronics”).
3️⃣ AI Extracts Attributes → GPT-4 Turbo extracts attributes (e.g., “Screen Size: 6.7 inches”).
4️⃣ Validation Engine Checks Data → Semantic Kernel detects inconsistencies in attributes.
5️⃣ Data is Stored → The PIM database saves validated product information.
Technology Stack (Proposed)
| Component | Suggested Technology |
|---|---|
| Data Extraction | Azure AI Document Intelligence |
| AI Models | Azure OpenAI (GPT-4, Embeddings) |
| Vector Storage | Qdrant, Azure Cognitive Search |
| Backend | .NET 9, ASP.NET Core, ML.NET |
| Orchestration | .NET Aspire, Azure Container Apps |
Conclusion & Next Steps
While DeepSeek’s AI proposal presents promising ideas, further analysis is needed to determine:
✅ Feasibility – Can these AI models handle Actualog’s complex data structures?
✅ Accuracy – How reliable are zero-shot classifications compared to supervised models?
✅ Scalability – How will .NET 9 & Azure AI perform under large-scale product ingestion?
We invite feedback from the Actualog community to refine this proposal. Which aspects do you think need the most improvement? 🚀
