JLT

Tag: AI Agents

  • Local-First AI: Running Language Models Without the Cloud

    Cloud-based AI is convenient. Upload your data, get results back, pay by the token. The model lives somewhere else, and so does your context. That trade-off works until it doesn’t.

    Running models locally changes the equation. Your data stays on your machine. Your context window belongs to you. Latency drops to milliseconds. Cost structure flips from per-token billing to one-time hardware investment.

    The Hardware Reality

    Local inference hardware has improved dramatically. A mid-range consumer laptop now runs 3-billion-parameter models in real time. Larger models, up to 70B parameters, run on desktop hardware with discrete GPUs or high-memory configurations.

    The Intel Core Ultra 9 185H, a laptop-class processor, handles 3B-8B parameter models at acceptable speeds without a discrete GPU. Adding a dedicated GPU shifts the ceiling significantly higher. The practical constraint isn’t hardware — it’s knowing which model fits your hardware and your task.

    What You Actually Gain

    Privacy is the obvious benefit. Code, documents, conversations — none of it leaves your machine. For enterprise users, this eliminates a category of compliance overhead. For individuals, it means your personal context isn’t training someone else’s model.

    Less discussed: latency changes how you interact with AI. When response times drop below 100ms, you stop treating AI as a separate workflow. It becomes part of your existing tools. The interaction model shifts from “submit prompt, wait, read response” to “iterate rapidly on ideas.”

    Offline capability matters more than it should. Presentations without wifi, flights, conference calls in venues with bad connectivity — the model still works. This isn’t theoretical. It changes which problems you attempt to solve with AI.

    The Trade-offs Are Real

    Smaller models have lower capability ceilings. A 3B parameter model won’t reason through complex multi-step problems the way a frontier model does. The gap closes for specific tasks — summarization, extraction, classification — but it doesn’t disappear.

    Maintenance overhead increases. Local models need updates, hardware upgrades, and troubleshooting. Cloud providers handle this invisibly. Self-hosting means you own the full stack.

    Context window management becomes your problem. Cloud providers abstract this away with retrieval-augmented generation or extended context windows. Running locally means you manage chunking, retrieval, and context overflow yourself.

    When It Makes Sense

    Local-first works when data sensitivity is high, when you need offline capability, or when usage volume would make cloud costs prohibitive. Development workflows with proprietary codebases fit this profile. Research workflows with sensitive documents fit it too.

    The sweet spot is tasks that don’t require frontier model capability. Summarization, extraction, classification, code completion — these work well at 3B-8B parameters. The moment you need multi-step reasoning on novel problems, cloud models still win.

    Most teams will end up using both. Local for privacy-sensitive, high-volume, latency-critical tasks. Cloud for capability-intensive tasks. The interesting question is how to build workflows that switch between them intelligently.

    What’s your current setup? Are you running models locally, or is everything cloud-based?

  • Why Every Developer Needs a Local AI Setup in 2026

    Six months ago, I recommended spinning up a VM before letting an AI agent loose on your system. It was good advice. But the landscape has shifted, and the recommendation has evolved.

    Running AI on someone else’s servers is fine for casual use. But if you’re a developer who writes code for a living — or even as a passionate hobby — you should seriously consider running at least some AI workloads on your own hardware. Here’s why.

    The Trust Equation Changed

    The Claude Code source code leak in March 2026 was a wake-up call for anyone who thought proprietary AI was a secure black box. When a single missed line in a configuration file can expose half a million lines of source code, including internal tooling, security logic, and hidden experimental features, it becomes clear that the “trust the provider” model has cracks.

    If a company as well-resourced as Anthropic can accidentally expose their entire codebase, what does that mean for the data you’re sending through their hosted APIs?

    Local models remove a variable from the trust equation. When the model runs on your machine, your data never leaves it. No terms of service to parse, no data usage policies to hope are enforced, no third-party server to get breached. What you type stays on your hardware. Full stop.

    It’s Easier Than You Think

    There’s a persistent myth that running AI locally requires a workstation that costs more than a used car. That was true two years ago. It isn’t anymore.

    You don’t need to train a model. You just need to run one — and for that, Ollama and llama.cpp have made the barrier to entry almost trivially low. On a modern laptop with 16GB of RAM and a decent CPU (no GPU required for smaller models), you can run a 7B or even 13B parameter model that handles code completion, summarization, drafting, and general Q&A quite well.

    The setup is usually: install Ollama, pull a model, and you’re done. No Docker, no CUDA (unless you want it), no venv hell. It takes about ten minutes.

    Not Everything Should Leave Your Machine

    Think about the tasks you do as a developer on any given day:

    • Pasting a stack trace to figure out what broke
    • Asking an AI to review a function before committing
    • Feeding it a config file to debug a deployment issue
    • Running it against your git diff to generate a commit message

    All of these involve code that might be proprietary, infrastructure details that reveal your architecture, or bugs that expose vulnerabilities. When you send these to a cloud API, you’re trusting that provider with information about your actual work product.

    With a local model, you can do all of this without transmitting a single byte externally. You can point the model at a codebase in your home directory and ask it things without creating a data trail. That’s not paranoia — it’s good operational hygiene.

    The Reality Check: Local Models Aren’t Magic

    Let’s be honest about what local models can and can’t do right now.

    A 7B model running locally won’t match GPT-4.5 on complex reasoning tasks. It won’t architect a microservices migration or catch subtle logic errors in your codebase. The smaller the model, the more you’re trading accuracy and depth for privacy and control.

    But here’s the thing: you don’t always need GPT-4.5. For code completion, docstring generation, regex writing, explaining errors, summarizing PRs, or drafting emails — small local models are genuinely competent. They’re good enough to save you hours of context-switching to the browser while keeping your work private.

    Think of it like having a junior colleague: they won’t design the system, but they’ll happily format your documentation, explain that cryptic error message, and write the boilerplate you really don’t want to type.

    When to Use Local vs Cloud

    The smartest approach isn’t “local only” or “cloud only.” It’s knowing which tool fits which job:

    Use local models for: Code review, debugging, writing scripts, generating documentation, experimenting, and anything involving sensitive code or data.

    Use cloud models for: Complex architecture decisions, multi-step reasoning, tasks requiring the latest knowledge, and anything that needs a frontier model to get right.

    This hybrid approach gives you the best of both: privacy and speed for the everyday grunt work, and raw power when the problem demands it.

    Getting Started

    If you’re curious, here’s the shortest path:

    • Install Ollama from ollama.com
    • Run ollama pull qwen2.5-coder:7b (a model specifically fine-tuned for code tasks)
    • Run ollama run qwen2.5-coder:7b and paste it some code

    That’s it. You now have a private AI coding assistant running on your own hardware. It won’t replace your cloud models, but it might surprise you with how much useful work it can do without ever phoning home.

    Have you tried running models locally yet? What’s the smallest model you’ve found that’s actually useful for your day-to-day work? Drop your setup in the comments.

  • The Real Reason Startups Are Firing Engineers and Hiring PMs (Or Vice Versa)

    If you’ve been paying attention to tech job postings lately, you’ve noticed a strange pattern. Some startups are quietly trimming their engineering teams — not the dramatic headlines of 30,000 cuts at Oracle, but slow, deliberate reductions. And at the same time, they’re hiring aggressively in product management, developer relations, and customer success.

    The obvious explanation is “AI will replace engineers.” It makes for a good tweet. But the reality is more interesting and more nuanced.

    The Cost-to-Value Equation Has Flipped

    Two years ago, a startup’s competitive advantage was its engineering velocity. If you could ship faster, iterate quicker, and build a more polished product than your competitors, you won. So startups hired engineers — lots of them. Every additional engineer meant more features, more experiments, more shipped code.

    AI has compressed that advantage. What used to take a team of three engineers a week now takes one engineer an afternoon with a capable AI coding assistant. The marginal value of each additional engineer has dropped, dramatically.

    But here’s the thing nobody talks about: building the product was always the easy part. Finding product-market fit, understanding what customers actually want, pricing it right, communicating it effectively, keeping customers happy — those things haven’t gotten any easier. If anything, AI has made them more important, because now everyone can build.

    The Real Bottleneck Moved

    In 2023, the bottleneck was engineering capacity. In 2026, it’s strategic clarity.

    A startup can now build a functioning MVP in a weekend. Three founders with AI assistants, no dedicated engineering team, and a clear vision can ship something that would’ve required six months and a $2M seed round two years ago. The barrier to building has collapsed.

    But the barrier to knowing what to build? That’s still incredibly hard.

    This is where the shift in hiring comes from. Startups are realizing that their scarcest resource isn’t coding capacity anymore — it’s product insight. They need people who can:

    • Talk to customers and translate messy, contradictory feedback into clear feature priorities
    • Define a positioning strategy that cuts through the noise of a thousand AI-wrapped competitors
    • Write PRDs that actually constrain AI behavior instead of vague wishlists
    • Design go-to-market motions that don’t rely on “build it and they will come”

    That’s a product manager’s job. It always has been. It just got way more valuable relative to everything else.

    But Here’s the Twist: It Goes Both Ways

    Not every startup is the same, and the reverse trend is equally real: engineering-heavy startups are finding they don’t need traditional PMs anymore.

    Why? Because a good engineer with an AI assistant can now do most of what a PM used to do. Draft a PRD? AI can help. Analyze user feedback? AI can summarize thousands of reviews in seconds. Create user personas? AI can do it from your existing customer data. Write a competitive analysis? Ten minutes with an LLM and a clear prompt.

    The PM role is getting squeezed from both sides. On one end, AI-augmented engineers are absorbing the tactical PM work (writing specs, prioritizing backlogs, analyzing data). On the other end, PMs who learn to use AI are becoming so efficient at their core work that fewer of them are needed.

    The surviving PMs are the ones who’ve moved up the value chain — from writing tickets to shaping strategy, from backlog management to market positioning, from feature spec to business model.

    What This Means for You

    If you’re an engineer: your coding skills are table stakes now. The engineers who thrive in 2026 are the ones who combine technical depth with product instinct. You need to be able to talk to users, understand market dynamics, and make judgment calls about what to build — not just how to build it.

    If you’re a PM: stop being a ticket factory. If your job is just writing user stories and grooming backlogs, you are one AI prompt away from obsolescence. Move toward strategy, toward user research, toward the parts of the job that require actual human judgment about what the market wants and why.

    The startups that will win in this environment are the ones that figure out the right ratio. Too many engineers without product direction means you’re building efficiently in the wrong direction. Too many PMs without building capacity means you’re strategizing with nothing to ship.

    The sweet spot is a small, sharp team of T-shaped people — engineers who understand their customers, and PMs who understand the technical tradeoffs — all operating at maximum leverage with AI doing the heavy lifting on execution.

    The org chart is flattening. The roles are blurring. And the people who’ll thrive are the ones who stop thinking about what their title is and start thinking about what the product needs.

    What do you think? Has your team’s ratio shifted, or are you seeing the opposite trend? I’m genuinely curious what the data looks like on the ground.

  • From ‘Chatbot’ to ‘Colleague’: Designing UX for AI Agents

    We’ve spent the last decade designing chatbots. They live in little bubbles on our screens, waiting for us to ask a question. But the next generation of AI isn’t just a chatbot—it’s a colleague. And designing the user experience (UX) for an AI that can take actions, edit files, and make decisions is a completely different challenge.

    The Trust Deficit

    When a chatbot gives you a wrong answer, it’s annoying. When an AI agent takes the wrong action—like deleting the wrong database or sending an email to the wrong person—it’s catastrophic. The primary goal of “Agentic UX” is to bridge the trust deficit between human intent and machine execution.

    This requires a shift from “chat” interfaces to “approval” interfaces. Instead of just showing a text response, the UI must clearly outline: What am I about to do? What tools will I use? And what is the potential risk?

    Designing for “Human-in-the-Loop”

    The most successful AI agents will be those that know when to pause. This is the concept of Human-in-the-Loop (HITL) design. A good agent UX should:

    • Show Its Work: Display a “thought process” or a step-by-step plan before executing complex tasks.
    • Provide Granular Permissions: Allow users to say “Yes” to reading a file but “No” to deleting it.
    • Offer Easy “Undo” Buttons: Since AI is probabilistic, mistakes will happen. The UI must make it easy to roll back changes.

    The Future of Interaction

    We are moving away from simple “prompt and response” toward a more collaborative “review and refine” workflow. As developers and product designers, our job is to make sure that while the AI is doing the heavy lifting, the human remains the pilot, not just a passenger.

    What’s the biggest frustration you’ve had with an AI agent so far? Was it a lack of transparency or a lack of control? Let’s discuss in the comments.

  • The ‘Agentic’ Workflow: How AI is Changing Product Requirements

    For decades, the Product Requirements Document (PRD) has been the bible of product development. It’s a static artifact—a Word doc or a Confluence page—that outlines what we’re building, for whom, and why. But as we shift from building traditional software to designing AI Agents, the humble PRD is undergoing a radical transformation.

    From Static Text to Dynamic Logic

    In a traditional workflow, a PRD describes a feature: “The user clicks a button, and the system generates a report.” In an agentic workflow, the requirements must account for autonomy and probability. We aren’t just defining a path; we’re defining a “solution space.”

    An AI-native spec doesn’t just say what the output should be; it defines the guardrails the agent must stay within. It includes:

    • Success Metrics as Code: Instead of “high accuracy,” we define specific evaluation datasets and pass/fail thresholds for the model.
    • Tool Selection Logic: A map of which APIs or databases the agent is allowed to touch and under what conditions.
    • Edge-Case Simulations: A list of “adversarial” inputs we expect the agent to handle without hallucinating or breaking.

    The Rise of the “Executable” PRD

    We are moving toward a world where the PRD is an executable file. Imagine a specification that not only tells the engineering team what to build but also serves as the initial “system prompt” or “evaluation harness” for the AI model itself. This shifts the PM’s role from “documenter” to “architect of behavior.”

    For product managers, this means learning to speak the language of constraints. It’s less about writing long paragraphs of user stories and more about defining the logical boundaries within which an intelligent agent can operate safely and effectively.

    Why This Matters for Your Career

    If you’re a PM looking to transition into AI, your ability to write these “agentic specs” will be your most valuable skill. It demonstrates that you understand not just the user’s intent, but the model’s limitations. It’s the difference between building a feature that “works sometimes” and one that users can actually trust.

    How are you adapting your product documentation for AI? Are you still using traditional PRDs, or have you moved to more dynamic frameworks? Let’s talk about it in the comments.

  • Qwen 3.6-Plus: A New Era for AI Agents

    Following the successful launch of the Qwen 3.5 series earlier this year, Alibaba has just dropped its latest powerhouse: Qwen 3.6-Plus. If you’ve been following the AI space, you know that each incremental update brings something new, but this one feels like a genuine leap forward—especially if you’re into building AI agents or doing complex coding tasks.

    What’s New in Qwen 3.6-Plus?

    Available right now via the Alibaba Cloud Model Studio API, Qwen 3.6-Plus isn’t just a minor tweak. It’s designed to be the engine behind “real-world agents.” Here are the big-ticket items that have the community buzzing:

    • Agentic Coding on Steroids: Whether you’re fixing a frontend bug or tackling a massive, repository-level architectural change, Qwen 3.6-Plus has been tuned to handle it with impressive accuracy. It’s built to “vibe code” alongside you, handling terminal operations and automated tasks like a seasoned engineer.
    • 1 Million Token Context Window: Yes, you read that right. By default, the model can process a massive amount of information at once. This is a game-changer for developers who need to feed entire codebases or massive technical manuals into the AI without losing the thread.
    • Sharper Multimodal Reasoning: It doesn’t just “see” images or charts; it understands them with much higher accuracy. This makes it incredibly reliable for tasks that involve interpreting complex diagrams or scientific data.

    Why It Matters for Developers

    The biggest hurdle with previous AI models was often their ability to stay on track during long, multi-step tasks. Qwen 3.6-Plus addresses this by deeply integrating reasoning, memory, and execution. In benchmarks like SWE-bench and Terminal-Bench 2.0, it’s matching or even surpassing industry leaders.

    For the average developer, this means less time babysitting the AI and more time seeing it actually do the work. It’s a move toward “highly autonomous super-agents” that can handle cross-domain planning and complex code management without constant human hand-holding.

    The “Vibe Coding” Experience

    Alibaba explicitly mentions that this release is designed to deliver a transformative “vibe coding” experience. It’s about making the interaction with AI feel more natural, stable, and reliable. By addressing feedback from the Qwen 3.5-Plus deployment, they’ve smoothed out the rough edges, making it a solid foundation for the next generation of AI-powered apps.

    Final Thoughts

    With Qwen 3.6-Plus, Alibaba is making a clear statement: the future of AI isn’t just about chatbots; it’s about agents that can actively participate in the development process. If you’re a developer looking to speed up your workflow or just curious about the cutting edge of open-weight models, Qwen 3.6-Plus is definitely worth a spin.

    Have you tried it out yet? Let me know how it handles your latest coding challenges!

Powered by
►
Necessary cookies enable essential site features like secure log-ins and consent preference adjustments. They do not store personal data.
None
►
Functional cookies support features like content sharing on social media, collecting feedback, and enabling third-party tools.
None
►
Analytical cookies track visitor interactions, providing insights on metrics like visitor count, bounce rate, and traffic sources.
None
►
Advertisement cookies deliver personalized ads based on your previous visits and analyze the effectiveness of ad campaigns.
None
Powered by