LLM 101 for Construction Professionals

You're hearing about AI everywhere.

It's gonna write your emails, analyze your specs, maybe even estimate your jobs someday.

But when you actually try to use some of these tools, like ChatGPT or tools built on similar tech (Large Language Models or LLMs), the results can be... underwhelming.

Sometimes it nails it, sometimes it completely misses the mark.

Why?

Often, it comes down to understanding the basic mechanics – the nuts and bolts of how these things actually "think" and process information. If you understand the engine, you can give it the right fuel and instructions to get where you need to go.

Let's cut through the jargon and look at four key pieces: Tokens, Prompts, Context Window, and Output.

1. Tokens: The AI's Building Blocks (Not Words!)

First thing to get straight: LLMs don't read words like you do. They read tokens.

• What's a Token? Think of it like a common chunk of text – often less than a word. A simple word like "estimate" might be one token, but a longer or less common word like "scaffolding" might be broken into "scaffold" and "ing" (two tokens). Punctuation like commas or periods are usually tokens, too. Even spaces can matter.
• The Rule of Thumb: On average, 1 token is roughly ¾ of a word in English. So, 100 tokens ≈ 75 words. A typical paragraph might be 100-150 tokens.
• Why It Matters:
  • Cost: Many AI services charge based on the number of tokens you process (both input and output). More tokens = higher cost. Efficiency matters.
  • Limits: AI models have limits on how much text they can handle at once (we'll get to that). Understanding tokens helps you gauge how much information you can feed in.
  • Example: "Change Order #123 requires approval" isn't 5 words, it might be 7 or 8 tokens because "#123" and "approval" get split.

Just remember: Tokens are the real currency the AI deals in.

2. Prompts: Your Instructions to the AI

A prompt is simply what you type into the AI – your question, your command, the instructions you give it.

• Garbage In, Garbage Out (GIGO): This is critical. The quality of the AI's output depends directly on the quality of your prompt.
  • Vague Prompt: "Summarize this safety report." -> Result: Might give you a generic, useless summary.
  • Specific Prompt: "Summarize this safety report, focusing only on corrective actions required for the excavation work on Site B, and list them as bullet points." -> Result: Much more likely to give you exactly what you need.
• Think Like You're Instructing a New Hire: Be clear, specific, and provide necessary context. Tell it the role it should play ("Act as a project manager reviewing this RFI response..."), the format you want ("Output as a JSON object," "Use numbered lists"), and the constraints ("Keep the summary under 100 words," "Do not include pricing information").
• Why It Matters: Your prompt is your steering wheel. A well-crafted prompt guides the AI precisely; a lazy prompt lets it wander off course. Mastering prompt writing (often called "prompt engineering") is key to getting value from LLMs.

3. Context Window: The AI's Working Memory

The context window is maybe the most important concept to grasp for practical use. Think of it as the AI's short-term memory or its workbench.

• What It Is: It's the maximum number of tokens the AI can "see" or consider at one time. This includes your prompt and the AI's response so far in a conversation.
• Size Matters (and Varies): Different models have different context window sizes:
  • Older models might have had 4,000 tokens (~3,000 words).
  • GPT-4 has variants up to 128,000 tokens (~100,000 words – like a decent-sized book).
  • Claude 3 models have up to 200,000 tokens (~150,000 words).
  • Meta's new Llama 4 models are pushing towards millions of tokens.
• The "Forgetting" Mechanism: When the conversation or the document you're analyzing exceeds the context window size, the AI starts to "forget" the earliest tokens. Imagine that workbench – you can only fit so much material on it. To add new stuff, you have to push old stuff off. The AI literally cannot see the tokens that have fallen out of the window.
• Why It Matters (Especially for Construction):
  • Big Documents: A 100-page specification document or a full set of drawings might contain way more tokens than even a large context window can hold. You can't just dump the whole thing in and expect the AI to understand it all perfectly.
  • Long Conversations: If you're having a back-and-forth with the AI about a complex issue, it might eventually "forget" details or instructions from the beginning of the chat.
  • Solutions: This limitation forces strategies like:
    • Chunking: Breaking large documents into smaller, relevant pieces to feed the AI.
    • Summarization: Having the AI summarize sections as you go, feeding the summary back in instead of the raw text.
    • RAG (Retrieval-Augmented Generation): Using a search system to find the most relevant chunks of information from your documents to put into the context window just when needed (like we discussed for the proposal writer).
    • Prioritization: Putting the most critical information directly in the prompt or close to where you need the output generated.

Understanding the context window limit is key to knowing why an AI might fail on large tasks and how to structure your approach to work around it.

4. Output: The Result You Get

The output is simply what the AI generates based on your prompt and the context it was given.

• Dependent on Everything Else: The quality, relevance, accuracy, and format of the output are directly determined by:
  • The clarity and specificity of your Prompt.
  • The information available (and not available) within the Context Window.
  • The underlying capabilities and training of the specific LLM you're using.
• Iterative Process: Don't expect the perfect output on the first try. Often, you'll need to refine your prompt, provide more context (or less, if it's noisy), or ask follow-up questions to steer the AI toward the desired result.
• Why It Matters: The output is the deliverable. If it's wrong, incomplete, or in the wrong format, the AI wasn't useful. Focusing on improving your prompts and managing the context is how you improve the output.

Why Should a Contractor Care?

Understanding these four pieces helps you move from randomly poking at AI to strategically using it as a tool:

• When analyzing a long spec document (Context Window), you know you need a tool or technique (like RAG) that can handle its size, not just paste it into a basic chatbot.
• When asking for a summary of daily reports (Prompt), you know to be specific about what information you need extracted and in what format (Output).
• When drafting proposal sections (Context & Prompt), you know you need to provide relevant examples from your past projects (Context) and clear instructions (Prompt) to get a useful first draft (Output).
• When estimating API costs (Tokens), you know that concise prompts and focused context save money.

These AI models are powerful pattern-matching machines with impressive language skills, but they aren't magic. They operate based on the instructions (prompt) and data (context) you provide, measured in their fundamental currency (tokens), all within the limits of their working memory (context window).

By understanding these basics, you're better equipped to leverage AI effectively, turning it from a novelty into a practical tool that can actually save you time and improve your work in the complex world of construction.