Saturday, August 29, 2026

Retrieval Is the Whole Game: Building AI Apps on Azure’s Data Stack in 2026

# Retrieval Is the Whole Game: Building AI Apps on Azure's Data Stack in 2026 Most AI projects that stall don't stall on the model. They stall about six weeks in, when someone asks why the assistant confidently cited a policy document that was superseded in 2023, and the team realizes the answer lives four layers down in a chunking script nobody owns. The Azure AI story over the last year has been a slow admission of exactly that. The interesting announcements haven't been about bigger models — they've been about the plumbing between your data and the model. If you're building on Azure right now, that plumbing is where the platform has changed most, and where your architecture decisions actually matter. Here's what the stack looks like today, what's genuinely production-ready, and where the sharp edges still are. ## First, the naming Azure AI Foundry became Microsoft Foundry at Ignite 2025. It's the same platform, extended: model catalog, agent hosting, evaluation, observability, and the SDK surface for all of it. You'll still see "Azure AI Foundry" in older docs, blog posts, and half the Stack Overflow answers you'll hit. Assume they're describing the same product unless something contradicts that. More consequentially, Azure AI Search now fronts as **Foundry IQ**. This one isn't just a rename — Foundry IQ is a new abstraction layer built on top of Azure AI Search, and understanding the relationship between them is most of what you need to know about retrieval on Azure in 2026. ## The four layers you're actually assembling Strip the branding and an Azure AI application in 2026 is four things: 1. **Models** — Foundry Models, a catalog rather than a single endpoint. GPT-5.2 and Codex Max went GA in the December–January wave; Claude reached general availability in Foundry in June 2026, hosted on Azure with the Messages API, prompt caching, and extended thinking. Open-weight options (Llama, Qwen, Mistral) are available both serverless and on managed compute. 2. **Retrieval** — Foundry IQ knowledge bases, or your own pipeline on Azure AI Search indexes. 3. **Data** — Cosmos DB for operational state and vectors, Fabric/OneLake for analytics, plus whatever's already in SharePoint and Blob Storage. 4. **Orchestration** — Foundry Agent Service, or Microsoft Agent Framework if you want to hold the reins yourself. The layer that most teams over-build is #2 and most teams under-build is #3. Let's take them in that order. ## Retrieval: stop rebuilding the pipeline The pattern Microsoft is targeting with Foundry IQ is one you've probably lived through. Every new agent gets its own retrieval stack — data connections, chunking logic, embedding model choice, index schema, permission filtering, query rewriting. Five agents later you have five subtly different pipelines, five sets of stale embeddings, and no consistent way to answer "why did it say that?" Foundry IQ inverts this. You define a **knowledge base** around a topic — employee policies, product docs, incident history — and any number of agents connect to it. Behind the knowledge base sit **knowledge sources**: Blob Storage, OneLake, SharePoint, existing Azure AI Search indexes, the public web via Bing, and MCP servers. For indexed sources, Foundry IQ runs the full ingestion pipeline itself — chunking, vectorization, hybrid index preparation — and keeps it synced. The genuinely new part is what happens at query time. Single-shot RAG (one query, one index, one hop) falls apart on ambiguous or multi-part questions. Foundry IQ's agentic retrieval engine treats retrieval as a reasoning task: - The incoming request is decomposed into subqueries - The engine selects which knowledge sources each subquery should hit - Subqueries fan out across local indexes and remote sources in parallel - Results are scored and reranked with the semantic ranker - If relevance is poor, it iterates — provided you've configured sufficient retrieval reasoning effort Microsoft's own numbers put the relevance improvement at around 36% over conventional single-shot RAG. Treat vendor benchmarks as directional, not gospel, but the architectural argument holds independent of the number: multi-hop questions need multi-hop retrieval. Permissions are handled at the retrieval layer rather than bolted on. Document-level access control flows through, and Purview sensitivity labels are respected through indexing and retrieval. If you've ever hand-rolled security trimming into an index filter, you know how much that's worth. ### When not to use it Foundry IQ is a good default. It is not always the right answer. Skip it if you have hard latency budgets — agentic retrieval means multiple model calls before you've generated a single token of the actual answer. Skip it if your retrieval is genuinely simple: one index, well-formed queries, known-good chunking. And skip it if you need chunking strategies it doesn't expose, because you're trading control for convenience by design. The GA story also deserves care. Parts of Foundry IQ are generally available through the Azure AI Search REST API version `2026-04-01`. Several of the more attractive capabilities — answer synthesis, higher reasoning effort levels, some source kinds — still require `2026-05-01-preview`. The portal experiences are preview across the board. Pin your API version explicitly and know which side of that line each feature you depend on sits. ## Operational data: Cosmos DB earned its place The two-database pattern — operational store plus a dedicated vector database — was the default for a long while, and it's now much harder to justify on Azure. Cosmos DB stores embeddings as a property alongside the rest of the document. The product record carries name, price, category, stock status, description, and vector in one item. Vector indexing uses DiskANN for approximate nearest-neighbor search at scale, and semantic reranking arrived in preview at Build 2026. You get hybrid search — vector plus filters plus the ordinary query predicates you were already writing — in one round trip, against data that's already consistent because it never left. Three additions from Build 2026 matter more for daily work than the headline features: - **The Linux emulator went GA.** Local build-test-validate on Linux, macOS, and Windows with no cloud dependency. This is the unglamorous fix that makes CI pipelines and onboarding stop being painful. - **An agent memory toolkit** standardizes persistent agent memory across Cosmos DB, Azure Durable Functions, and Foundry models — worth reading before you design your own conversation store. - **`langchain-azure-cosmosdb`** consolidates vector search, chat memory, and semantic caching behind LangChain and LangGraph interfaces, if that's your orchestration layer. A rough heuristic: if the vectors describe entities your application already stores and mutates, keep them in Cosmos DB. If they describe a document corpus that's ingested and rarely changes, put them behind Azure AI Search and let Foundry IQ manage the pipeline. ## Analytics: mirroring instead of ETL Fabric's contribution to AI apps is mostly about not moving data twice. Mirror Cosmos DB into Fabric and operational data lands in OneLake in near-real time with no ETL job to own, no schedule to babysit, and no drift between what the app sees and what the analytics layer sees. Fabric SQL and Cosmos DB in Fabric write to OneLake by default. The payoff for AI work is that OneLake is a first-class Foundry IQ knowledge source. Your curated lakehouse tables become groundable knowledge without a separate export path. If you're already invested in Fabric, this is the shortest route from governed enterprise data to a grounded agent. ## Agents and memory Foundry Agent Service has been converging on the OpenAI Responses protocol, which is a meaningful simplification if you've been maintaining separate code paths. Build 2026 added hosted runtimes, Toolboxes (public preview) for connecting agents to external tools, and expanded memory — procedural, user, and session — in the Foundry Agent Service memory store, which handles extraction, consolidation, and retrieval across sessions automatically. Agents can also publish directly into Microsoft Teams and Microsoft 365 Copilot. If your users live in those surfaces, that distribution path removes a lot of frontend work. For tool connectivity there's a cloud-hosted Foundry MCP server at `mcp.ai.azure.com` with Entra auth, connectable from VS Code, Visual Studio, or the Foundry portal — no local process to manage. ## The SDK situation, plainly This is the part that will bite you in a code review, so it's worth being blunt about. `azure-ai-projects` v2 is now the canonical SDK for everything Foundry. Agents, inference, evaluations, and memory are unified in one package — the separate `azure-ai-agents` dependency is gone, and `openai` plus `azure-identity` come bundled as direct dependencies. Active development happens on the beta line. Pin your versions. Separately: the Azure Machine Learning SDK v1 reached end of support on June 30, 2026. The CLI v1 extension went a year earlier. If you still have v1-based training pipelines running, they're unsupported now, and the migration to v2's YAML-first job definitions is not a weekend task. Budget for it properly. ## How I'd start a new project today 1. **Inventory before you architect.** Where does the knowledge actually live, and who's allowed to see it? Permission modeling is the thing that kills pilots at the production gate, not retrieval quality. 2. **Start with one Foundry IQ knowledge base** and the simplest retrieval reasoning effort that gets acceptable answers. Turn it up only against a measured baseline. 3. **Keep operational vectors in Cosmos DB**, document vectors behind Azure AI Search. Don't run two vector stores because two blog posts told you to. 4. **Instrument evaluation from day one.** Foundry's evaluation tooling can convert agent traces into evaluation datasets — build that loop before you have opinions to defend, not after. 5. **Write down which API versions you depend on** and which of them are preview. Then set a calendar reminder to recheck, because this surface is moving fast enough that a six-month-old architecture decision deserves a second look. The platform has gotten substantially better at the boring parts — ingestion, permissions, memory, evaluation. That's genuinely good news, because the boring parts were always where the projects died. What hasn't changed is that grounding quality is a data problem wearing an AI costume. No amount of retrieval reasoning effort will fix a knowledge base full of documents nobody has curated since 2021.

Thursday, August 27, 2026

How to Actually Learn AI

How to Actually Learn AI

How to Actually Learn AI

Most advice about learning AI is useless. It's either a reading list nobody finishes, or a course that teaches you to build a spam classifier you'll never think about again.

Here's what has worked for people I've watched go from zero to competent.

Start with something you want to exist

Not a tutorial. A thing. A tool that summarizes your saved articles. A script that sorts your photos. A bot that answers questions about your own notes.

The project matters because it decides what you learn next. When you're stuck on a real problem, the next thing to read is obvious. When you're working through a curriculum, everything looks equally important, so nothing sticks.

Pick something small enough to finish in a weekend, and slightly harder than you're comfortable with.

Use the models before you study them

You don't need to know how a transformer works to build useful things with one. Spend your first month just calling APIs and seeing what happens.

You'll learn more about how these systems behave from a hundred bad outputs than from a paper explaining attention. You'll develop intuitions: what they're good at, where they hallucinate, how much the prompt matters, when a bigger model is worth it and when it isn't.

That intuition is the actual skill. The theory makes more sense afterward anyway, because you'll have real questions instead of abstract ones.

Learn Python properly

You can avoid this for a while. Eventually you won't want to.

You don't need to be a great programmer. You need to be comfortable enough that code isn't the bottleneck when you're trying to think about something else. Loops, functions, dictionaries, reading error messages without panicking, installing packages, and enough pandas to move data around.

A week of focused practice gets most people there.

Then go one layer down

Once you've built a few things, the black box starts to bother you. Good — that's when theory lands.

The order that tends to work:

  • How neural networks train. Gradient descent, loss, backpropagation. Do this once with actual code, not just diagrams.
  • What embeddings are and why they're everywhere.
  • How transformers work, at least well enough to explain attention to someone else.
  • How models get fine-tuned, and why that's different from prompting.

Andrej Karpathy's videos are the standard recommendation for a reason. He builds things from scratch in front of you, which is different from being told how they work.

Read papers badly

Papers feel intimidating because people assume you're supposed to read them start to finish and understand everything. Nobody does that.

Read the abstract. Look at the figures. Read the conclusion. If it still seems relevant, go back for the method. Most papers you'll abandon halfway, and that's the correct outcome.

Do this once a week and after a few months you'll notice you can follow conversations that used to be opaque.

Build in public, or at least in front of one person

Write up what you made. Post it somewhere. Explain it to a friend who doesn't work in tech.

This is not about self-promotion. It's that explaining forces you to notice the parts you only half-understand. You'll be halfway through a sentence and realize you have no idea why the thing works. That gap is the most useful thing you'll find all week.

Ignore most of the news

The field produces an enormous amount of noise. New model releases, benchmark arguments, predictions about the next five years. Almost none of it changes what you should do tomorrow.

Follow a few people who build things rather than comment on things. Check in weekly, not hourly. The fundamentals move slowly, and they're where your time pays off.

What this looks like over six months

Month one: build something small with an API. It'll be ugly. Finish it anyway.

Months two and three: get comfortable with Python and data handling. Build two or three more things, each slightly harder.

Months four and five: go under the hood. Train a small model yourself. Understand what you were calling all this time.

Month six: pick a direction. Applications, research, infrastructure, evaluation — they're different jobs. You'll know by then which one interests you, and that's a decision you couldn't have made on day one.

The uncomfortable part

You'll spend a lot of time confused. Not the productive kind of confused where you're one insight away — the kind where you're not sure what question to even ask.

That's normal, and it doesn't mean you're behind. It's what learning something genuinely new feels like from the inside. Everyone who seems fluent went through the same stretch and just didn't post about it.

Keep building. The confusion clears.

Wednesday, August 26, 2026

How Developers Are Actually Using AI Coding Assistants Day to Day (Beyond Autocomplete)

How Developers Are Actually Using AI Coding Assistants Day to Day (Beyond Autocomplete)

AI, Software Engineering, Productivity

A couple of years ago, "AI coding assistant" basically meant autocomplete that finished your line of code. That's no longer the whole story. Tools like GitHub Copilot, Claude, and Cursor have quietly changed how a lot of day-to-day development work actually gets done — not by replacing developers, but by taking over specific chunks of the workflow. Here's a practical look at what that actually looks like in practice, not the marketing version.

1. Inline autocomplete — still the baseline

This is the layer most developers started with, and it's still useful for exactly what it sounds like: finishing a function signature, writing boilerplate (getters/setters, repetitive test cases, standard error handling), or predicting the next few lines based on a pattern you've already established in the file.

It's fast and low-friction, but it's also the least "intelligent" layer — it's pattern-matching on your current context, not reasoning about your codebase.

2. Chat-based pair programming

This is where things get more useful. Instead of asking for the next line, you describe a problem and get a discussion, not just code:

"I have a race condition in this async function when two requests
hit it concurrently. Here's the function — what's actually happening,
and what are two different ways to fix it?"

Used well, this replaces a chunk of what used to be "let me search Stack Overflow" or "let me ask a senior dev on the team." The key difference from autocomplete: you're getting an explanation you can evaluate, not just code you have to trust blindly.

3. Agentic, multi-step tasks

The newest layer — tools like Claude Code and Cursor's agent mode can take a task description, explore your actual codebase, make changes across multiple files, run your tests, and iterate based on the results, largely unsupervised for a stretch.

This is genuinely different from the first two categories. It's less "smart autocomplete" and more "hand off a scoped task and review the diff." It works best on well-defined, bounded tasks — "add input validation to this form and write tests for it" — and works worse on vague, judgment-heavy ones like "make this codebase better."

4. Code review and explanation

A quieter but high-value use case: pointing an AI tool at a pull request or an unfamiliar file and asking "what does this actually do, and what would you flag in review?" It won't replace a human reviewer's judgment about whether a change is the right call for the product, but it's very good at catching the mechanical stuff — an unhandled edge case, an inconsistent naming pattern, a missing null check — before a human even looks at it.

5. Where these tools still fall short

Worth being honest about this, since overclaiming is how teams end up disappointed:

- They don't know your team's unwritten conventions unless you tell them.
- They can be confidently wrong about library APIs or version-specific behavior.
- Multi-file agentic changes still need a real review — treat the diff
  the same way you'd treat a junior dev's PR, not a senior dev's.
- They're worse at architecture-level judgment calls than at
  bounded, well-specified tasks.

6. Practical tips to get more out of them

A few things that consistently make a real difference in output quality:

- Give context, not just instructions: paste the relevant function/file,
  not just a description of the problem.
- Ask for the reasoning, not just the code, when the problem is subtle
  — it makes it much easier to catch mistakes.
- For agentic tasks, scope them tightly ("fix this one bug and add a
  regression test") rather than broadly ("improve error handling").
- Always review the diff like you would a colleague's PR — these tools
  are a strong first draft, not a final answer.

Conclusion

The realistic picture is less "AI writes the code now" and more "AI has absorbed a chunk of the busywork and first-draft thinking that used to eat up a developer's day." The teams getting the most out of it aren't the ones treating it as magic — they're the ones that have figured out which of the four layers above fits which kind of task, and where a human still needs to be the one making the call.

 

Struggling with slow queries or a messy .NET codebase?
I help teams fix SQL Server performance issues and clean up .NET/Angular applications. If you're stuck on something similar to what's in this post, contact me — happy to take a look.

Tuesday, July 8, 2025

Intelligent Query Processing (IQP)

 

Intelligent Query Processing (IQP)

(Especially enhanced in SQL Server 2019–2025)


🔍 What it is:

IQP is a suite of features that automatically improves the performance of your queries without changing your code.


🚀 Why it’s the best:

  • Fixes bad plans on the fly with Adaptive Joins, Memory Grant Feedback, and Parameter Sensitivity fixes.

  • ✅ Great for complex, slow-running queries—IQP helps optimize execution without developer tuning.

  • ✅ In SQL Server 2025, IQP is even smarter and works better with modern workloads, JSON data, and AI features.


🧠 Simple Example:

Normally, if you write a query with a weird data shape or a bad parameter, SQL Server might guess wrong and run it slowly. IQP learns from that and adjusts future runs to be faster—all automatically.

Wednesday, May 21, 2025

SQL Learning: A Step-by-Step Guide for Beginners

SQL Learning: A Step-by-Step Guide for Beginners

Learning SQL (Structured Query Language) is an essential skill for anyone working with databases. This step-by-step guide will take you from the absolute basics to more advanced concepts, with examples to help you practice along the way.

Phase 1: Understanding Databases and SQL Fundamentals

Step 1: What is a Database?

  • Definition: A database is an organized collection of structured data stored electronically
  • RDBMS: Relational Database Management Systems (like SQL Server, MySQL, PostgreSQL) organize data into tables
  • Tables: Collections of related data organized in rows and columns
  • Relationships: How tables connect to each other (one-to-one, one-to-many, many-to-many)

Step 2: SQL Basics

  • What is SQL: Structured Query Language - the standard language for interacting with databases
  • SQL dialects: Different database systems (SQL Server, MySQL, Oracle, etc.) have slight variations
  • SQL operations: CRUD operations (Create, Read, Update, Delete)

Step 3: Setting Up Your Learning Environment

  • Choose a database system: For beginners, SQLite, MySQL, or SQL Server Express are good options
  • Install a database client: Tools like MySQL Workbench, SQL Server Management Studio, or DBeaver
  • Create your first database:
CREATE DATABASE LearnSQL;
USE LearnSQL;

Phase 2: Creating and Manipulating Database Objects

Step 4: Creating Tables

  • Basic syntax:
CREATE TABLE Customers (
    CustomerID INT PRIMARY KEY,
    FirstName VARCHAR(50) NOT NULL,
    LastName VARCHAR(50) NOT NULL,
    Email VARCHAR(100),
    Phone VARCHAR(20)
);
  • Data types: Common types include INT, VARCHAR, DATE, DECIMAL, BOOLEAN
  • Constraints: Rules applied to data columns (PRIMARY KEY, FOREIGN KEY, NOT NULL, UNIQUE, DEFAULT)

Step 5: Modifying Tables

  • Adding columns:
ALTER TABLE Customers
ADD Address VARCHAR(200);
  • Modifying columns:
ALTER TABLE Customers
ALTER COLUMN Phone VARCHAR(30);
  • Deleting columns:
ALTER TABLE Customers
DROP COLUMN Address;

Step 6: Creating Relationships Between Tables

  • Foreign keys:
CREATE TABLE Orders (
    OrderID INT PRIMARY KEY,
    CustomerID INT,
    OrderDate DATE,
    TotalAmount DECIMAL(10,2),
    FOREIGN KEY (CustomerID) REFERENCES Customers(CustomerID)
);

Phase 3: Basic Data Manipulation

Step 7: Inserting Data

  • Single row insert:
INSERT INTO Customers (CustomerID, FirstName, LastName, Email)
VALUES (1, 'John', 'Smith', 'john.smith@example.com');
  • Multiple row insert:
INSERT INTO Customers (CustomerID, FirstName, LastName, Email)
VALUES 
    (2, 'Jane', 'Doe', 'jane.doe@example.com'),
    (3, 'Robert', 'Johnson', 'robert.j@example.com');

Step 8: Basic Queries with SELECT

  • Selecting all columns:
SELECT * FROM Customers;
  • Selecting specific columns:
SELECT FirstName, LastName, Email FROM Customers;
  • Using WHERE for filtering:
SELECT * FROM Customers WHERE LastName = 'Smith';

Step 9: Updating Data

  • Basic update:
UPDATE Customers
SET Email = 'new.email@example.com'
WHERE CustomerID = 1;
  • Multiple column update:
UPDATE Customers
SET Email = 'updated@example.com', Phone = '555-1234'
WHERE CustomerID = 2;

Step 10: Deleting Data

  • Deleting specific rows:
DELETE FROM Customers WHERE CustomerID = 3;
  • Safety practice: Always test with SELECT before DELETE:
-- First run this to check what will be deleted
SELECT * FROM Customers WHERE LastName = 'Johnson';
-- Then run the delete if the results match expectations
DELETE FROM Customers WHERE LastName = 'Johnson';

Phase 4: Intermediate Querying

Step 11: Advanced Filtering with WHERE

  • Comparison operators: =, <>, <, >, <=, >=
SELECT * FROM Orders WHERE TotalAmount > 100;
  • Logical operators: AND, OR, NOT
SELECT * FROM Customers 
WHERE (LastName = 'Smith' OR LastName = 'Doe')
AND NOT Email IS NULL;
  • BETWEEN operator:
SELECT * FROM Orders
WHERE OrderDate BETWEEN '2023-01-01' AND '2023-12-31';
  • IN operator:
SELECT * FROM Customers
WHERE CustomerID IN (1, 3, 5);
  • LIKE operator (for pattern matching):
-- Find all customers whose last name starts with 'S'
SELECT * FROM Customers WHERE LastName LIKE 'S%';

-- Find all customers whose email contains 'example'
SELECT * FROM Customers WHERE Email LIKE '%example%';

Step 12: Sorting Results with ORDER BY

  • Basic sorting:
SELECT * FROM Customers ORDER BY LastName;
  • Multiple column sorting:
SELECT * FROM Customers ORDER BY LastName, FirstName;
  • Direction (ascending/descending):
SELECT * FROM Orders ORDER BY TotalAmount DESC;

Step 13: Limiting Results

  • SQL Server and PostgreSQL:
SELECT TOP 5 * FROM Customers;
  • MySQL and SQLite:
SELECT * FROM Customers LIMIT 5;

Step 14: Grouping Results with GROUP BY

  • Basic grouping:
SELECT CustomerID, COUNT(*) AS OrderCount
FROM Orders
GROUP BY CustomerID;
  • Common aggregate functions: COUNT, SUM, AVG, MIN, MAX
SELECT 
    CustomerID,
    COUNT(*) AS OrderCount,
    SUM(TotalAmount) AS TotalSpent,
    AVG(TotalAmount) AS AverageOrder,
    MIN(TotalAmount) AS SmallestOrder,
    MAX(TotalAmount) AS LargestOrder
FROM Orders
GROUP BY CustomerID;
  • Filtering groups with HAVING:
SELECT CustomerID, COUNT(*) AS OrderCount
FROM Orders
GROUP BY CustomerID
HAVING COUNT(*) > 5;

Phase 5: Joining Tables

Step 15: Inner Joins

  • Basic inner join:
SELECT c.FirstName, c.LastName, o.OrderID, o.OrderDate
FROM Customers c
INNER JOIN Orders o ON c.CustomerID = o.CustomerID;

Step 16: Outer Joins

  • Left outer join:
SELECT c.FirstName, c.LastName, o.OrderID, o.OrderDate
FROM Customers c
LEFT JOIN Orders o ON c.CustomerID = o.CustomerID;
  • Right outer join:
SELECT c.FirstName, c.LastName, o.OrderID, o.OrderDate
FROM Customers c
RIGHT JOIN Orders o ON c.CustomerID = o.CustomerID;
  • Full outer join:
SELECT c.FirstName, c.LastName, o.OrderID, o.OrderDate
FROM Customers c
FULL OUTER JOIN Orders o ON c.CustomerID = o.CustomerID;

Step 17: Self Joins

  • Joining a table to itself (Example with employees and managers):
CREATE TABLE Employees (
    EmployeeID INT PRIMARY KEY,
    Name VARCHAR(100),
    ManagerID INT
);

SELECT e.Name AS Employee, m.Name AS Manager
FROM Employees e
LEFT JOIN Employees m ON e.ManagerID = m.EmployeeID;

Phase 6: Subqueries and Advanced Concepts

Step 18: Subqueries

  • Subquery in WHERE:
SELECT * FROM Customers
WHERE CustomerID IN (
    SELECT DISTINCT CustomerID 
    FROM Orders 
    WHERE TotalAmount > 1000
);
  • Subquery in SELECT:
SELECT 
    c.CustomerID,
    c.FirstName,
    c.LastName,
    (SELECT COUNT(*) FROM Orders o WHERE o.CustomerID = c.CustomerID) AS OrderCount
FROM Customers c;

Step 19: Common Table Expressions (CTE)

  • Basic CTE:
WITH HighValueCustomers AS (
    SELECT CustomerID, SUM(TotalAmount) AS TotalSpent
    FROM Orders
    GROUP BY CustomerID
    HAVING SUM(TotalAmount) > 5000
)
SELECT c.FirstName, c.LastName, h.TotalSpent
FROM Customers c
JOIN HighValueCustomers h ON c.CustomerID = h.CustomerID;

Step 20: Views

  • Creating a view:
CREATE VIEW CustomerOrders AS
SELECT 
    c.CustomerID,
    c.FirstName,
    c.LastName,
    o.OrderID,
    o.OrderDate,
    o.TotalAmount
FROM Customers c
JOIN Orders o ON c.CustomerID = o.CustomerID;
  • Using a view:
SELECT * FROM CustomerOrders WHERE TotalAmount > 500;

Phase 7: Database Administration Basics

Step 21: Indexes

  • Creating an index:
CREATE INDEX idx_customer_lastname ON Customers(LastName);
  • Unique index:
CREATE UNIQUE INDEX idx_customer_email ON Customers(Email);

Step 22: Transactions

  • Basic transaction:
BEGIN TRANSACTION;

UPDATE Accounts SET Balance = Balance - 100 WHERE AccountID = 1;
UPDATE Accounts SET Balance = Balance + 100 WHERE AccountID = 2;

COMMIT TRANSACTION;
-- If something goes wrong: ROLLBACK TRANSACTION;

Step 23: Stored Procedures

  • Creating a simple stored procedure:
CREATE PROCEDURE GetCustomerOrders
    @CustomerID INT
AS
BEGIN
    SELECT * FROM Orders WHERE CustomerID = @CustomerID;
END;
  • Executing a stored procedure:
EXEC GetCustomerOrders @CustomerID = 1;

Phase 8: Practical Projects

Step 24: Build a Complete Database

Create a fully normalized database for a small business with multiple tables:

  • Customers
  • Products
  • Orders
  • OrderItems
  • Employees
  • Categories

Step 25: Write Complex Queries

Practice writing complex queries like:

  • Sales reports by region
  • Inventory tracking
  • Customer purchasing patterns
  • Employee performance metrics

Step 26: Database Maintenance Tasks

Learn practical maintenance:

  • Backup and restore
  • Check database integrity
  • Update statistics
  • Rebuild indexes

Learning Resources

Online Tutorials and Courses

  • W3Schools SQL Tutorial
  • SQLZoo
  • Khan Academy's SQL Course
  • Codecademy SQL Courses
  • SQL Bolt

Practice Platforms

  • LeetCode (Database section)
  • HackerRank SQL challenges
  • SQL Fiddle
  • DB Fiddle

Books

  • "SQL Queries for Mere Mortals" by John Viescas
  • "Learning SQL" by Alan Beaulieu
  • "SQL Cookbook" by Anthony Molinaro

Tips for Success

  1. Practice regularly: SQL is learned through regular practice
  2. Start with simple queries: Master the basics before moving to complex queries
  3. Use real-world scenarios: Try to solve problems related to real business needs
  4. Review and optimize: Regularly review your SQL code to find better ways to write queries
  5. Join SQL communities: Participate in forums and communities to learn from others
  6. Read documentation: Different database systems have different features, so check the documentation

Remember, becoming proficient in SQL takes time and practice. Start small, be patient, and gradually tackle more complex problems as your understanding improves.


Happy SQL Learning!

Thursday, May 15, 2025

SQL Server's Intelligent Query Processing Enhancements

 SQL Server's Intelligent Query Processing (IQP) framework has seen significant improvements in recent versions, particularly in SQL Server 2022 and the 2024 preview. These features collectively aim to improve query performance automatically without requiring code changes.

Key Intelligent Query Processing Enhancements

Parameter Sensitive Plan Optimization (PSP)

  • Addresses the "parameter sniffing" problem by automatically maintaining multiple cached execution plans for the same query
  • The query optimizer intelligently selects the most appropriate plan based on input parameter values
  • Reduces the need for manual query hints or plan guides
  • Particularly valuable for queries where different parameter values require drastically different execution strategies

Cardinality Estimation Improvements

  • Enhanced statistical modeling for more accurate row count predictions
  • Better handling of correlated columns and complex predicates
  • Improved histogram analysis for skewed data distributions
  • More accurate estimations for expressions and functions

Memory Grant Feedback

  • Adaptive memory allocation that learns from previous executions
  • Prevents both under-allocation (causing expensive spills to disk) and over-allocation (wasting memory)
  • Works for both batch and row mode execution
  • Now supports concurrent query executions and persists feedback across server restarts

Batch Mode on Rowstore

  • Extends batch processing benefits (traditionally limited to columnstore indexes) to rowstore tables
  • Automatically applied to eligible complex queries on traditional tables
  • Significant performance boosts for analytical queries without requiring index changes

Degree of Parallelism (DOP) Feedback

  • Dynamically adjusts parallelism based on past query execution metrics
  • Prevents both under-parallelization (not using available resources) and over-parallelization (wasting resources on coordination)
  • Adapts to varying server workloads

Interleaved Execution

  • Optimizes plans with multi-statement table-valued functions (MSTVFs)
  • Pauses optimization, executes parts of the query to get accurate cardinality, then resumes optimization
  • No longer requires explicit recompile hints

Query Store Improvements

  • Enhanced performance insights with more granular metrics
  • Better plan forcing mechanisms
  • Improved query wait statistics for bottleneck identification
  • Automatic plan correction capabilities for identifying and fixing regression-prone queries

These improvements work together as part of SQL Server's commitment to "it just works better" query processing, requiring minimal administrator intervention while delivering substantial performance benefits.

Tuesday, May 13, 2025

Microsoft Visual Studio 2010 install the NuGet Package Manager

 In Microsoft Visual Studio 2010, NuGet package management was not built-in by default—you have to install the NuGet Package Manager extension manually. Here's how you can do it:


🛠️ Step-by-Step: Install NuGet in Visual Studio 2010

  1. Open Visual Studio 2010.

  2. Go to:
    Tools > Extension Manager

  3. In the Extension Manager:

    • Click on Online Gallery

    • Search for "NuGet Package Manager"

    • Click Download and follow the installation prompts.

  4. Restart Visual Studio 2010 after installation.


📦 Adding a NuGet Package to a Project

After NuGet is installed:

  1. Right-click on your project in Solution Explorer

  2. Select "Manage NuGet Packages..."

  3. In the NuGet window:

    • Search for the package you want

    • Click Install