My LinkedIn Connection Cleared Cognizant’s GenAI Engineer Interview — 5 Deep Questions They Asked on Transformers & RAG
Q, K, V matrices. GELU vs ReLU. Chunking strategies. Multimodal embeddings. Retrieval methods. Here’s every question with the exact answers that worked.
Last week, Rahul sent me a message I wasn’t expecting.
“Bhai, Cognizant ki GenAI interview cleared kar li. Ek hi round tha but bohot deep tha. Questions share karta hoon — post kar dena.”
Rahul is a Senior Data Scientist with 7 years of experience, the last three focused on LLMs and RAG systems. He’d been following my GenAI interview content for months. When he cracked his GenAI Engineer interview at Cognizant, he wanted to pay it forward.
What surprised him most? “They didn’t ask basic definitions. Every question had a ‘why’ and a ‘when would you choose this over that.’ They wanted engineering thinking, not textbook answers.”
Here are all 5 questions — and exactly how he answered them.
What is the significance of Query (Q), Key (K), and Value (V) in the Transformer architecture? How do they work together in self-attention, and why are three different matrices needed?
Rahul said this question came within the first 3 minutes. The interviewer added: “Don’t just tell me what they are — tell me why we need all three and what breaks if we remove one.”
Think of self-attention as a soft search engine inside the model. Every token is simultaneously a searcher (Query), a candidate result (Key), and a piece of information to be retrieved (Value).
- Query (Q): What am I looking for? The current token’s “question” to the rest of the sequence.
- Key (K): What do I offer? Every other token’s “advertisement” of its content.
- Value (V): What do I actually contain? The real information retrieved when a Key matches a Query.
- Each token embedding is projected into three separate vectors: Q, K, V — using three learned weight matrices (W_Q, W_K, W_V).
- Attention score between token i and token j = dot product of Q_i and K_j, scaled by √d_k to prevent gradient saturation.
- Scores are passed through Softmax to get a probability distribution — how much should token i attend to token j?
- Final output = weighted sum of all Value vectors, where weights come from the Softmax scores.
This was the follow-up. Here’s Rahul’s answer that impressed the interviewer:
- If Q = K: The model would always attend most to tokens identical to itself — self-similarity would dominate, and the model would miss meaningful cross-token relationships.
- If K = V: What you “advertise” (Key) would be exactly what you “give” (Value). This removes the model’s ability to separate relevance signaling from information content — a key source of representational richness.
- Three separate matrices allow the model to independently learn: what to search for, what to match against, and what to actually retrieve. This separation is what makes attention flexible enough to learn complex language patterns.
Which activation functions are used in Transformers and why? Where are Softmax, ReLU, and GELU used — and why do BERT, GPT, and LLaMA prefer GELU over ReLU?
Rahul expected this to be a quick definition question. It wasn’t. The interviewer wanted the mathematical reasoning and practical trade-offs.
Used in: (1) The attention score normalization step and (2) the final output layer for next-token prediction.
Why: Softmax converts raw scores (logits) into a probability distribution that sums to 1. In attention, this tells the model how much weight to give each token. In the output layer, it tells the model the probability of each vocabulary token being the next word.
Used in: Feed-forward sublayer of earlier Transformer variants (e.g., the original “Attention Is All You Need” paper used ReLU).
Why ReLU works: Simple, fast, no vanishing gradient for positive inputs. Formula: f(x) = max(0, x)
Why ReLU falls short in Transformers:
- “Dying ReLU” problem — neurons with negative inputs output exactly 0 and stop learning.
- Hard, non-smooth cutoff at 0 — can cause training instabilities in very deep networks.
- No gradient for negative inputs — information is permanently lost.
Used in: Feed-forward sublayer in BERT, GPT-2, GPT-4, LLaMA, Gemini, and virtually all modern LLMs.
GELU (Gaussian Error Linear Unit) is a smooth, probabilistic activation function:
Why GELU beats ReLU for LLMs:
- Smooth gradient everywhere: Unlike ReLU’s hard zero cutoff, GELU tapers smoothly near zero — better gradient flow throughout training.
- Stochastic gating behavior: GELU can be interpreted as randomly dropping inputs (like dropout) based on how negative they are — a built-in regularization effect.
- Empirically superior: BERT, GPT-2, and subsequent models all showed measurably better performance with GELU vs ReLU on language benchmarks.
- Non-zero for negative inputs: GELU allows small negative values to pass through (unlike ReLU’s hard zero), preserving more nuanced gradient signal.
| Activation | Location in Transformer | Key Property | Limitation |
|---|---|---|---|
| Softmax | Attention scores + output layer | Converts to probability distribution | Computationally expensive at large vocab |
| ReLU | Feed-forward layer (older models) | Fast, simple, sparse | Dying neurons, hard cutoff |
| GELU | Feed-forward layer (BERT, GPT, LLaMA) | Smooth, stochastic gating | Slightly more compute than ReLU |
What are the different retrieval methods used in RAG? Explain BM25, Dense Retrieval, Hybrid Search, Parent-Child Retrieval, Multi-Query Retrieval, and Contextual Compression — and when would you use each?
Rahul said this was the question that separated him from other candidates. Most people know BM25 and dense retrieval. Few can articulate all six and defend their trade-offs.
What it is: A classic TF-IDF-based ranking algorithm that scores documents based on exact keyword frequency, penalizing very long documents.
When to use: When users use exact terminology — product codes, legal clause numbers, medical billing codes. Fast, no GPU needed, interpretable.
Limitation: Zero semantic understanding. “Heart attack” won’t match “myocardial infarction.”
What it is: Both query and documents are embedded into dense vectors. Retrieval is done via cosine/dot-product similarity in vector space.
When to use: When semantic similarity matters — paraphrased questions, multilingual content, domain-specific language. Handles synonyms and context naturally.
Limitation: Can miss exact keyword matches. Requires a good embedding model. Needs a vector database (Pinecone, ChromaDB, pgvector).
What it is: Combines BM25 (sparse) and dense retrieval results using a fusion algorithm like Reciprocal Rank Fusion (RRF). Each method retrieves top-k candidates independently, then rankings are merged.
When to use: Production systems where queries are unpredictable — some users type exact codes, others describe concepts. Consistently outperforms either method alone. Rahul’s default recommendation for enterprise RAG.
What it is: Documents are chunked at two levels — small child chunks (for precise semantic matching) and larger parent chunks (for rich context). Retrieval finds the best child chunk, then returns its parent for generation.
When to use: When context continuity matters — legal contracts, technical manuals, research papers. Small chunks give retrieval precision; large parent chunks give the LLM enough context to generate a complete answer.
Analogy: You search for a specific paragraph (child), but you read the whole section (parent) to understand it.
What it is: An LLM rewrites the original user query into multiple (typically 3–5) semantically varied versions. Each variant is used to retrieve chunks independently. Results are de-duplicated and merged.
When to use: When user queries are ambiguous, poorly worded, or multi-faceted. Dramatically improves recall by casting a wider semantic net. Particularly effective in customer support RAG where users often don’t know exactly what to search for.
What it is: Retrieved chunks are passed through a compressor LLM that extracts only the sentences or passages directly relevant to the query — removing noise before passing context to the generator.
When to use: When chunks are large and contain a mix of relevant and irrelevant content. Reduces token cost and improves answer quality by giving the generator a cleaner, more focused context.
| Method | Best For | Trade-off |
|---|---|---|
| BM25 | Exact keyword / code matching | No semantic understanding |
| Dense | Semantic / paraphrase matching | Misses exact terms |
| Hybrid | General production RAG | More complex to set up |
| Parent-Child | Long structured documents | Needs two-level indexing |
| Multi-Query | Ambiguous / multi-part queries | Higher LLM call cost |
| Contextual Compression | Noisy, large chunks | Extra LLM pass = added latency |
How do you create embeddings for documents containing text, tables, and images? Real-world enterprise documents are rarely just plain text.
This is where most candidates freeze. They know how to embed text. They’ve never thought about what to do with a PDF that has 3 tables, 2 charts, and 50 pages of mixed content.
Standard embedding models like text-embedding-3-small only accept text. A PDF with tables rendered as images, charts, and scanned pages is largely invisible to them. You’ll miss 40–60% of the information in a typical enterprise document.
Standard pipeline: extract text → chunk → embed with a text embedding model. But for scanned PDFs, use OCR (pytesseract, Google Cloud Vision, AWS Textract) first to convert image-based text into extractable strings before embedding.
Tables lose their structure when flattened to raw text. Two approaches Rahul uses:
- Table-to-text conversion: Use an LLM to convert each table into a natural language description. “The revenue for Q1 2024 was $4.2M, representing a 12% YoY increase…” This is embeddable as regular text and highly retrievable.
- Structured metadata tagging: Store table data as JSON alongside the embedding. The embedding captures semantic meaning; the JSON enables exact lookups when needed.
- Image captioning: Use a vision-language model (GPT-4o, LLaVA, BLIP-2) to generate a detailed text description of each image or chart. Embed the caption. The retrieval system can now find images based on semantic queries.
- Multimodal embeddings: Use models like CLIP (Contrastive Language-Image Pre-training) that embed both images and text into the same vector space. Enables direct image-text similarity search without an intermediate caption step.
Every chunk, regardless of modality, gets metadata attached: source document name, page number, section header, content type (text/table/image), last updated date, author. This metadata enables filtered retrieval — “only search page 5–10 of the Q3 report” — dramatically improving precision.
What are the different chunking strategies in RAG, and which one would you choose? Discuss the trade-offs between retrieval quality, context preservation, and computational cost.
Rahul said the interviewer was very specific: “Don’t just list them. Tell me when each one breaks and what you choose for production.”
How it works: Split document into chunks of exactly N tokens (e.g., 512 tokens), with optional overlap (e.g., 50 tokens).
Pros: Simple, fast, predictable chunk sizes, easy to implement.
Cons: Blindly cuts mid-sentence, mid-paragraph, or mid-table. Destroys semantic coherence. Retrieved chunks can start with “…therefore the contract clause is invalid” with no context of what the clause was.
Use when: Quick prototyping or when document structure is highly uniform.
How it works: Tries to split on natural boundaries in order: paragraph breaks → sentence breaks → word breaks → characters. Falls back to the next level only if the chunk is still too large.
Pros: Respects natural language structure much better than fixed-size. Default in LangChain’s RecursiveCharacterTextSplitter.
Cons: Still doesn’t understand semantic meaning — a paragraph boundary isn’t always a topic boundary.
Use when: General-purpose RAG with unstructured text documents.
How it works: Embed every sentence. Find split points where cosine similarity between adjacent sentences drops significantly — a topic boundary. Group semantically similar sentences into one chunk.
Pros: Chunks are semantically coherent — each chunk covers one topic. Best retrieval quality of all text-based methods.
Cons: Computationally expensive (requires embedding every sentence during indexing). Variable chunk sizes can make context window management tricky.
Use when: High-quality RAG where retrieval precision is critical and indexing time is not a constraint.
How it works: Fixed-size chunks with a large overlap (e.g., 256-token chunks with 128-token overlap). Each chunk shares content with its neighbors.
Pros: Ensures no information is stranded at a chunk boundary. Every key sentence appears in at least two chunks.
Cons: High redundancy — significantly more chunks to store and retrieve. Higher indexing and storage cost.
Use when: Dense technical content (legal contracts, scientific papers) where every sentence might be critical.
How it works: Index small child chunks (e.g., 128 tokens) for precise retrieval. Store larger parent chunks (e.g., 512 tokens). When a child chunk is retrieved, return the parent chunk to the LLM.
Pros: Combines retrieval precision with generation context richness. Best of both worlds.
Cons: Two-level indexing adds architectural complexity. Requires careful parent-child mapping.
Use when: Long structured documents where precise retrieval and rich generation context are both needed.
How it works: Use the document’s inherent structure — chapters, sections, headers, bullet points — as chunk boundaries. Each section becomes one chunk.
Pros: Perfectly preserves document semantics. Each chunk is a complete logical unit. Extremely interpretable — easy to cite sources.
Cons: Requires structured documents (with headers, markdown, XML). Doesn’t work on unstructured plain text.
Use when: Well-structured knowledge bases — product manuals, SOPs, wikis, HTML documentation.
| Strategy | Retrieval Quality | Context Preserved | Compute Cost | Best For |
|---|---|---|---|---|
| Fixed-Size | Low | Poor | Very Low | Prototyping |
| Recursive | Medium | Good | Low | General text RAG |
| Semantic | High | Excellent | High | Precision-critical RAG |
| Sliding Window | Medium-High | Good | Medium | Dense technical docs |
| Parent-Child | High | Excellent | Medium | Long structured docs |
| Structure-Based | High | Excellent | Low | SOPs, manuals, wikis |
What Cognizant’s GenAI Interview Is Really Testing
When I debriefed with Rahul, he said one thing stuck with him: “The interviewer never asked me to define anything. Every question was ‘explain the trade-offs’ or ‘when would you choose this over that.’ They wanted to see how I think, not what I’ve memorized.”
That’s the signal. Cognizant at the GenAI Engineer level wants engineers who can design production systems, defend their choices, and understand why things break at scale.
- Know the math behind Q, K, V. Not just “Query, Key, Value” but why three separate matrices, and what each dimension of separation enables.
- GELU vs ReLU is not trivia. The smooth gradient flow and stochastic behavior of GELU is a real engineering reason that shows up in benchmark results. Know the formula and the “why.”
- Retrieval methods are a decision matrix. Learn all six, but more importantly — learn when each one fails and what you reach for instead.
- Multimodal documents are the new normal. If you can’t handle tables and images in your RAG pipeline, you’re not production-ready for enterprise clients.
- Chunking is your first design decision. It determines everything downstream. Never treat it as an afterthought.
Did this help your Cognizant or GenAI interview prep? Give it 50 claps — it helps this reach more engineers who need it.
Have you faced a deep technical GenAI interview recently? DM me on LinkedIn. I’ll feature your story next.
Follow me for a new company interview breakdown every single day.