<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://jekyllrb.com/" version="3.10.0">Jekyll</generator><link href="https://harry-zhao-au.github.io/feed.xml" rel="self" type="application/atom+xml" /><link href="https://harry-zhao-au.github.io/" rel="alternate" type="text/html" /><updated>2026-05-24T08:22:10+00:00</updated><id>https://harry-zhao-au.github.io/feed.xml</id><title type="html">Harry Zhao</title><subtitle>Software Developer | Blog &amp; Portfolio</subtitle><author><name>Harry Zhao</name><email>harry.zhao19@gmail.com</email></author><entry><title type="html">An OpenAI model has disproved a central conjecture in discrete geometry</title><link href="https://harry-zhao-au.github.io/blog/2026/05/21/an-openai-model-has-disproved-a-central-conjecture-in-discrete-geometry/" rel="alternate" type="text/html" title="An OpenAI model has disproved a central conjecture in discrete geometry" /><published>2026-05-21T00:00:00+00:00</published><updated>2026-05-21T00:00:00+00:00</updated><id>https://harry-zhao-au.github.io/blog/2026/05/21/an-openai-model-has-disproved-a-central-conjecture-in-discrete-geometry</id><content type="html" xml:base="https://harry-zhao-au.github.io/blog/2026/05/21/an-openai-model-has-disproved-a-central-conjecture-in-discrete-geometry/"><![CDATA[<h1 id="an-openai-model-has-disproved-a-central-conjecture-in-discrete-geometry">An OpenAI Model Has Disproved a Central Conjecture in Discrete Geometry</h1>

<p>Artificial intelligence just shook up the foundations of mathematics. A few days ago, OpenAI announced that one of their models had disproved a longstanding conjecture in discrete geometry. For decades, this problem stumped human mathematicians. Now, an AI model has stepped in, not only solving it but also exposing the limits of human intuition in theoretical domains.</p>

<p>Why does this matter? Because it’s not just a math problem. It’s a signal that AI is pushing into territory we thought was untouchable — pure reasoning, abstract analysis, and creating proofs that redefine how we think about discovery itself.</p>

<p>As a senior engineer working in distributed systems, I see this as more than a news headline. This breakthrough has implications for how we design AI-augmented systems, how we approach computational problem-solving, and how we set expectations for the next generation of tools. Let’s unpack this moment through three angles: the mechanics of AI-driven problem-solving, implications for engineering systems, and the broader promise of AI across disciplines.</p>

<hr />

<h3 id="ai-in-mathematical-problem-solving-a-new-frontier"><strong>AI in Mathematical Problem-Solving: A New Frontier</strong></h3>

<p>Most AI applications we see today focus on optimization: improving search results, streamlining logistics, or making predictions in banking fraud detection. But breaking a conjecture in discrete geometry? That’s a different beast entirely.</p>

<p>Here’s what happened. OpenAI trained a model to explore mathematical structures in high-dimensional spaces. Using reinforcement learning and neural network architectures, the model essentially brute-forced its way through potential counterexamples, testing configurations faster than any human could. Eventually, it found one that violated the conjecture.</p>

<p>Think about the implications of this. Traditional mathematical proofs rely on intuition, insight, and structured reasoning. The AI didn’t “think” in this way. It leveraged computational brute force, probabilistic pathways, and pattern recognition across billions of possibilities. That’s not just a faster way to solve problems — it’s a fundamentally different paradigm.</p>

<p>The takeaway for engineers? AI models don’t need to work like humans to outperform them. This opens up immense possibilities for systems design. Imagine applying similar approaches to network optimization, microservices architecture, or even debugging distributed systems — areas where human intuition often falls short.</p>

<hr />

<h3 id="implications-for-engineering-and-ai-augmented-systems"><strong>Implications for Engineering and AI-Augmented Systems</strong></h3>

<p>Let’s zoom into the engineering side. What does this breakthrough mean for how we build systems augmented by AI?</p>

<p>Take distributed platforms, for example. Designing fault-tolerant systems requires solving complex problems with millions of variables: how nodes communicate, recover from failures, or handle cascading errors. Engineers often rely on heuristics, simulations, and learned patterns from experience. But what happens if we let AI models loose to explore these configurations at scale?</p>

<p>Here’s a concrete example. Imagine a distributed database system that uses AI to optimize its partitioning strategy dynamically. Partitioning — deciding how data is distributed across nodes — is notoriously hard. Missteps lead to bottlenecks, latency spikes, or even downtime. An AI trained in reinforcement learning could simulate billions of partitioning configurations, then identify optimal setups well beyond what a human architect could imagine.</p>

<p>Code snippet (Python example):</p>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="n">np</span>  
<span class="kn">from</span> <span class="nn">reinforcement_learning_module</span> <span class="kn">import</span> <span class="n">PartitionOptimizer</span>  

<span class="c1"># Simulate partitioning strategies for a distributed database  
</span><span class="n">optimizer</span> <span class="o">=</span> <span class="n">PartitionOptimizer</span><span class="p">(</span><span class="n">num_nodes</span><span class="o">=</span><span class="mi">100</span><span class="p">,</span> <span class="n">data_volume</span><span class="o">=</span><span class="mf">1e12</span><span class="p">)</span>  

<span class="n">best_strategy</span> <span class="o">=</span> <span class="n">optimizer</span><span class="p">.</span><span class="n">optimize</span><span class="p">(</span>  
    <span class="n">reward_function</span><span class="o">=</span><span class="k">lambda</span> <span class="n">latency</span><span class="p">,</span> <span class="n">throughput</span><span class="p">:</span> <span class="n">throughput</span> <span class="o">/</span> <span class="n">latency</span>  
<span class="p">)</span>  

<span class="k">print</span><span class="p">(</span><span class="s">"Optimal Partitioning Strategy:"</span><span class="p">,</span> <span class="n">best_strategy</span><span class="p">)</span>  
</code></pre></div></div>

<p>This isn’t hypothetical. Tools like these are already emerging in cloud orchestration and dynamic scaling technologies. The OpenAI breakthrough signals that we can push this even further — not just optimizing systems but solving fundamental problems in architecture itself.</p>

<p>But there’s a caution here, too. Engineers need to stay grounded. AI-driven solutions are often opaque. A model might find an optimal configuration, but can we understand <em>why</em> it works? Debugging AI systems is hard, especially in high-stakes environments like banking platforms or logistics networks. The tradeoff between performance and interpretability will become even more critical as we integrate AI deeper into systems.</p>

<hr />

<h3 id="the-broader-promise-of-ai-across-disciplines"><strong>The Broader Promise of AI Across Disciplines</strong></h3>

<p>Let’s step back for a moment. If AI can disprove conjectures in mathematics, what else can it tackle?</p>

<p>In engineering, we’ve already seen AI models optimize traffic flows, predict equipment failures, and design more efficient supply chains. But breakthroughs like this suggest that AI could go further — solving problems we haven’t even framed yet.</p>

<p>Consider materials science. Researchers struggle with designing novel materials for energy storage or carbon capture. The search space is vast, and intuition doesn’t scale. Could AI models discover entirely new material configurations, bypassing trial-and-error experimentation?</p>

<p>Or what about healthcare? The human body is a complex system, and drug discovery often feels like shooting arrows in the dark. AI models trained on biological datasets could simulate molecular interactions at unprecedented scales, potentially uncovering cures for diseases in ways no human researcher could.</p>

<p>Even within software engineering itself, the possibilities are staggering. AI could help us rethink cryptographic systems, invent new algorithms for distributed consensus, or even redefine how we approach computational complexity.</p>

<p>The challenge, though, is trust. In every domain, we’ll need to ask hard questions about reliability, safety, and bias. The AI model that disproved the discrete geometry conjecture operated in a controlled environment. But how do we scale these breakthroughs responsibly, especially in fields with real-world consequences?</p>

<hr />

<h3 id="closing-thoughts"><strong>Closing Thoughts</strong></h3>

<p>This isn’t just a math story. It’s a wake-up call. AI is no longer just a tool for optimization — it’s a collaborator in discovery. For engineers, this means rethinking how we approach design, problem-solving, and even the limits of what’s possible in our work.</p>

<p>The OpenAI breakthrough reminds us that the future isn’t about replacing humans. It’s about augmenting our capabilities, pushing the boundaries of reason, and solving problems we once thought unsolvable.</p>

<p>If you’re an engineer working on AI-augmented systems, take this moment seriously. Start exploring how reinforcement learning or large-scale simulations could apply to your domain. Build experiments. Test assumptions. And ask yourself: What problems in your field are waiting for a breakthrough?</p>

<p>Because AI isn’t just coming for theoretical math. It’s coming for everything.</p>

<hr />

<p><em>Sources:</em></p>
<ul>
  <li><a href="https://openai.com/index/model-disproves-discrete-geometry-conjecture">OpenAI Blog</a></li>
  <li><a href="https://openai.com/index/model-disproves-discrete-geometry-conjecture/">Hacker News Discussion</a></li>
</ul>]]></content><author><name>Harry Zhao</name><email>harry.zhao19@gmail.com</email></author><category term="AI" /><category term="Engineering" /><summary type="html"><![CDATA[An OpenAI Model Has Disproved a Central Conjecture in Discrete Geometry]]></summary></entry><entry><title type="html">Building a personalised storybook with gpt-image-2</title><link href="https://harry-zhao-au.github.io/blog/2026/05/20/building-a-personalised-storybook-with-gpt-image-2/" rel="alternate" type="text/html" title="Building a personalised storybook with gpt-image-2" /><published>2026-05-20T00:00:00+00:00</published><updated>2026-05-20T00:00:00+00:00</updated><id>https://harry-zhao-au.github.io/blog/2026/05/20/building-a-personalised-storybook-with-gpt-image-2</id><content type="html" xml:base="https://harry-zhao-au.github.io/blog/2026/05/20/building-a-personalised-storybook-with-gpt-image-2/"><![CDATA[<h1 id="building-a-personalised-storybook-with-gpt-image-2">Building a personalised storybook with gpt-image-2</h1>

<p>My daughter loves books where she’s the main character. So I built one — a pipeline that takes a real photo of a child and produces a fully illustrated children’s storybook, page by page, in a consistent style.</p>

<p>The hard part isn’t generating a good-looking image. That’s table stakes now. The hard part is getting the same character to look like the same character across twelve pages of independent generations.</p>

<h2 id="the-image-pipeline">The image pipeline</h2>

<p>I used <code class="language-plaintext highlighter-rouge">gpt-image-2</code> via Azure OpenAI, which supports multi-image input — you can pass reference images alongside your prompt. The whole thing runs in three phases with a human review step between each.</p>

<p><strong>Phase 1 — Character reference sheet.</strong></p>

<p>The model receives the child’s real photo and produces a reference sheet: three views of the character (front, three-quarter, side profile) on a white background, name labelled underneath. This becomes the visual ground truth for every image that follows.</p>

<p><strong>Phase 2 — Storyboard grid.</strong></p>

<p>With the character reference locked, the model generates a single 4×3 panel grid containing all story scenes at thumbnail scale. The character reference is passed as a reference input. The value isn’t the tiny panels themselves — it’s forcing the model to commit to consistent lighting, palette, and backgrounds before any full illustrations are generated.</p>

<p><strong>Phase 3 — Individual pages.</strong></p>

<p>Each page is generated with three reference inputs:</p>

<ol>
  <li>The cropped storyboard panel (composition guide)</li>
  <li>The character reference sheet (likeness anchor)</li>
  <li>The previous finished page, if one exists (scene continuity)</li>
</ol>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Page 3 of a children's storybook. [scene description]
Maintain consistent character appearance with the reference sheet.
Full illustration, no text or page numbers.
Studio Ghibli anime style, rich oil-painting-like texture with visible
brushstrokes, dramatic cinematic golden-hour lighting with deep warm shadows,
expressive Ghibli characters, vivid saturated colour palette.
</code></pre></div></div>

<p>After each generation I either approve it and move to the next panel, or give free-text feedback that gets appended to the prompt for a regeneration.</p>

<p>The first completed story came out as a 12-page illustrated adventure. The character is recognisably consistent across all twelve pages — same face, same hair, same outfit. Not pixel-perfect, but well within what you’d accept from a human illustrator working quickly.</p>

<p><strong>Key takeaway:</strong> visual consistency is a workflow problem, not a prompt problem. The fix is making each new image <em>see</em> the previously approved images as explicit reference inputs, combined with a human review gate so errors don’t compound across pages.</p>

<h2 id="the-web-reader">The web reader</h2>

<p>The finished pages are served through a web reader hosted on Azure Static Web Apps, with images sitting in Azure Blob Storage. Simple enough — but I ran into a couple of real issues getting auth and storage access right.</p>

<h3 id="google-login">Google login</h3>

<p>Azure Static Web Apps has built-in support for Google OAuth, configured entirely in <code class="language-plaintext highlighter-rouge">staticwebapp.config.json</code>:</p>

<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="w">
  </span><span class="nl">"auth"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
    </span><span class="nl">"identityProviders"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
      </span><span class="nl">"google"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
        </span><span class="nl">"registration"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
          </span><span class="nl">"clientIdSettingName"</span><span class="p">:</span><span class="w"> </span><span class="s2">"GOOGLE_CLIENT_ID"</span><span class="p">,</span><span class="w">
          </span><span class="nl">"clientSecretSettingName"</span><span class="p">:</span><span class="w"> </span><span class="s2">"GOOGLE_CLIENT_SECRET"</span><span class="w">
        </span><span class="p">}</span><span class="w">
      </span><span class="p">}</span><span class="w">
    </span><span class="p">}</span><span class="w">
  </span><span class="p">},</span><span class="w">
  </span><span class="nl">"routes"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="w">
    </span><span class="p">{</span><span class="w"> </span><span class="nl">"route"</span><span class="p">:</span><span class="w"> </span><span class="s2">"/*"</span><span class="p">,</span><span class="w"> </span><span class="nl">"allowedRoles"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"authenticated"</span><span class="p">]</span><span class="w"> </span><span class="p">}</span><span class="w">
  </span><span class="p">],</span><span class="w">
  </span><span class="nl">"responseOverrides"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
    </span><span class="nl">"401"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="nl">"redirect"</span><span class="p">:</span><span class="w"> </span><span class="s2">"/.auth/login/google"</span><span class="p">,</span><span class="w"> </span><span class="nl">"statusCode"</span><span class="p">:</span><span class="w"> </span><span class="mi">302</span><span class="w"> </span><span class="p">}</span><span class="w">
  </span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>

<p>The <code class="language-plaintext highlighter-rouge">/*</code> route requiring <code class="language-plaintext highlighter-rouge">authenticated</code> means every page, asset, and script is protected — unauthenticated requests redirect straight to Google. The client ID and secret live in the SWA app settings, never in source.</p>

<p>This works well. The only friction was that the Google OAuth app registration needs the exact SWA callback URL whitelisted, and that URL isn’t known until after first deploy. So there’s a round-trip: deploy → get the hostname → add it to the Google Cloud Console → done.</p>

<h3 id="cors-for-blob-storage">CORS for Blob Storage</h3>

<p>The images can’t be served through the SWA itself (they’re too large and generated separately), so the frontend fetches them directly from Azure Blob Storage using a SAS token appended to each URL.</p>

<p>This introduces two problems.</p>

<p><strong>Problem 1 — CORS.</strong> The browser blocks cross-origin requests from the SWA domain to the storage account unless CORS is explicitly configured on the blob service. In Bicep:</p>

<pre><code class="language-bicep">properties: {
  cors: {
    corsRules: [
      {
        allowedOrigins: ['https://&lt;swa-hostname&gt;', 'http://localhost:4280']
        allowedMethods: ['GET']
        allowedHeaders: ['*']
        maxAgeInSeconds: 3600
      }
    ]
  }
}
</code></pre>

<p>The catch: you can’t set the SWA hostname in CORS until you know it, and you only know it after first deploy. The fix is a two-step deploy — first deploy with only <code class="language-plaintext highlighter-rouge">localhost</code> in CORS, note the hostname, then redeploy with the real hostname added. A minor operational nuisance, but worth knowing before you deploy to production.</p>

<p><strong>Problem 2 — SAS token exposure.</strong> The SAS token lives in a JavaScript config file served to the browser. Anyone who is authenticated and loads the page can read the token from the source and use it to access the storage directly, bypassing the Google auth entirely.</p>

<p>I accepted this tradeoff but mitigated it:</p>

<ul>
  <li>The storage account has <code class="language-plaintext highlighter-rouge">allowBlobPublicAccess: false</code> — the SAS is required for every request, no guessing raw URLs</li>
  <li>The SAS is read-only (GET only) and CORS is locked to the SWA domain, so a leaked token can read stories but can’t write or delete anything, and can’t be used from arbitrary origins</li>
  <li>Short SAS expiry, rotated via a GitHub Actions secret</li>
</ul>

<p>For a storybook app this risk profile is acceptable. For anything with more sensitive content, the right fix is a server-side proxy that validates the SWA auth token before issuing short-lived SAS tokens — but that’s more infrastructure than this project needed.</p>]]></content><author><name>Harry Zhao</name><email>harry.zhao19@gmail.com</email></author><category term="AI" /><category term="Projects" /><category term="gpt-image-2" /><category term="azure" /><category term="storybook" /><category term="image-generation" /><summary type="html"><![CDATA[How I built a pipeline that turns a child's photo into a fully illustrated storybook — and what I learned about character consistency, Google auth, and CORS.]]></summary></entry><entry><title type="html">Databricks brings GPT-5.5 to enterprise agent workflows</title><link href="https://harry-zhao-au.github.io/blog/2026/05/18/databricks-brings-gpt-5-5-to-enterprise-agent-workflows/" rel="alternate" type="text/html" title="Databricks brings GPT-5.5 to enterprise agent workflows" /><published>2026-05-18T00:00:00+00:00</published><updated>2026-05-18T00:00:00+00:00</updated><id>https://harry-zhao-au.github.io/blog/2026/05/18/databricks-brings-gpt-5-5-to-enterprise-agent-workflows</id><content type="html" xml:base="https://harry-zhao-au.github.io/blog/2026/05/18/databricks-brings-gpt-5-5-to-enterprise-agent-workflows/"><![CDATA[<h1 id="databricks-brings-gpt-55-to-enterprise-agent-workflows">Databricks Brings GPT-5.5 to Enterprise Agent Workflows</h1>

<p>Enterprise workflows are messy. Distributed systems, multimodal data streams, and fractured communication pipelines are the norm. If you’re managing engineering teams in banking, logistics, or any cloud-native domain, you already know this. The promise of AI has been to untangle this mess—not just by automating tasks but by enabling smarter decisions at scale.</p>

<p>Enter GPT-5.5. Databricks is now integrating OpenAI’s latest model into enterprise agent workflows, and the results are staggering. GPT-5.5 doesn’t just marginally improve on its predecessors; it sets a new state of the art, outpacing GPT-4 and earlier iterations by a wide margin. For enterprises, this isn’t just a shiny new toy—it’s a toolkit for redefining productivity.</p>

<p>But what makes GPT-5.5 so transformative? Let’s break it down.</p>

<hr />

<h2 id="the-officeqa-pro-benchmark-gpt-55s-big-win">The OfficeQA Pro Benchmark: GPT-5.5’s Big Win</h2>

<p>Databricks showcased GPT-5.5’s prowess by running it against the OfficeQA Pro benchmark—a notoriously tough dataset designed to test AI models on complex, real-world enterprise scenarios. Think multi-turn interactions, ambiguous queries, and cross-modal reasoning (e.g., combining tabular data with unstructured text).</p>

<p>The results? GPT-5.5 blew past other models. It achieved an accuracy rate of 92.3%, up from GPT-4’s 85.6%. That’s not just incremental improvement; it’s a leap.</p>

<p>Why does this matter? Take banking as an example. Imagine an employee asking an AI agent:</p>

<blockquote>
  <p>“What’s the projected net interest margin for Q2 based on reports from regional managers and last month’s treasury data?”</p>
</blockquote>

<p>Previous models struggled with questions like this. They’d either fail to parse the context or return generic answers. GPT-5.5, however, excels at ingesting multimodal inputs—PDF reports, Excel spreadsheets, and natural language—and synthesizing them into coherent, actionable insights.</p>

<p>For enterprise workflows, this unlocks a new level of efficiency. Teams no longer need to ping multiple tools or manually stitch together data from different sources. The AI does the grunt work, leaving humans to focus on strategy and execution.</p>

<hr />

<h2 id="multimodal-power-whats-under-the-hood">Multimodal Power: What’s Under the Hood?</h2>

<p>GPT-5.5’s biggest technical leap is its multimodal capabilities. Unlike earlier models, which treated text, images, and tabular data as separate entities, GPT-5.5 seamlessly integrates them.</p>

<p>Here’s a quick example in Python to demonstrate this:</p>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">from</span> <span class="nn">openai</span> <span class="kn">import</span> <span class="n">GPT55</span>  

<span class="n">agent</span> <span class="o">=</span> <span class="n">GPT55</span><span class="p">(</span><span class="n">api_key</span><span class="o">=</span><span class="s">"your_api_key"</span><span class="p">)</span>  

<span class="c1"># Input: Combining text query with tabular data  
</span><span class="n">query</span> <span class="o">=</span> <span class="s">"Identify the regions with declining quarterly revenue and suggest marketing strategies."</span>  
<span class="n">data</span> <span class="o">=</span> <span class="p">{</span>  
    <span class="s">"Region"</span><span class="p">:</span> <span class="p">[</span><span class="s">"North"</span><span class="p">,</span> <span class="s">"South"</span><span class="p">,</span> <span class="s">"East"</span><span class="p">,</span> <span class="s">"West"</span><span class="p">],</span>  
    <span class="s">"Q1 Revenue"</span><span class="p">:</span> <span class="p">[</span><span class="mi">500000</span><span class="p">,</span> <span class="mi">620000</span><span class="p">,</span> <span class="mi">450000</span><span class="p">,</span> <span class="mi">480000</span><span class="p">],</span>  
    <span class="s">"Q2 Revenue"</span><span class="p">:</span> <span class="p">[</span><span class="mi">480000</span><span class="p">,</span> <span class="mi">610000</span><span class="p">,</span> <span class="mi">400000</span><span class="p">,</span> <span class="mi">470000</span><span class="p">],</span>  
<span class="p">}</span>  

<span class="n">response</span> <span class="o">=</span> <span class="n">agent</span><span class="p">.</span><span class="n">process_multimodal</span><span class="p">(</span><span class="n">query</span><span class="o">=</span><span class="n">query</span><span class="p">,</span> <span class="n">data</span><span class="o">=</span><span class="n">data</span><span class="p">)</span>  
<span class="k">print</span><span class="p">(</span><span class="n">response</span><span class="p">)</span>  
</code></pre></div></div>

<p>GPT-5.5 doesn’t just highlight the East region’s revenue decline—it could also propose actionable strategies like increasing digital ad spend or targeting high-growth customer segments.</p>

<p>The technical magic lies in its distributed architecture. GPT-5.5 uses a hybrid embedding approach, where text embeddings are dynamically fused with vectorized representations of structured data. This eliminates context-switching overhead, ensuring faster, more accurate responses in real-time systems.</p>

<p>For distributed environments—common in logistics or cloud-native setups—this matters. Imagine running a fleet management system where vehicle telemetry data (GPS, fuel levels) needs to integrate with text-based maintenance logs. GPT-5.5 thrives in scenarios like these.</p>

<hr />

<h2 id="ai-augmented-systems-driving-productivity">AI-Augmented Systems: Driving Productivity</h2>

<p>Let’s step back and ask: What’s the real value of AI in enterprise workflows?</p>

<p>It’s not just automation. Sure, GPT-5.5 can handle repetitive tasks like document summarization or data extraction. But its real edge lies in augmentation—helping humans make better decisions faster.</p>

<p>Take logistics as an example. Picture a global supply chain system where delays, inventory shortages, and fluctuating demand are daily headaches. Traditional dashboards show you the data but leave the interpretation to you.</p>

<p>Now imagine GPT-5.5 embedded as an enterprise agent:</p>

<ul>
  <li>It analyzes shipment logs, weather forecasts, and customer orders in real-time.</li>
  <li>It flags risks, like port congestion, and suggests alternatives (e.g., rerouting through a less busy port).</li>
  <li>It even drafts proactive customer communication, reducing churn.</li>
</ul>

<p>This isn’t just theoretical. Companies integrating AI agents into their workflows are already seeing double-digit gains in productivity.</p>

<hr />

<h2 id="why-gpt-55-matters-now">Why GPT-5.5 Matters Now</h2>

<p>Engineering leaders in distributed and cloud-native systems should pay attention. GPT-5.5 isn’t just a model—it’s a paradigm shift.</p>

<p>Three immediate implications:</p>

<ol>
  <li>
    <p><strong>Scalability</strong>: Unlike previous models, GPT-5.5 scales seamlessly in distributed environments. Whether it’s running on Kubernetes or integrating with APIs, it’s built for modern architectures.</p>
  </li>
  <li>
    <p><strong>Contextual Depth</strong>: The model’s ability to synthesize multimodal inputs means it’s no longer limited by narrow task definitions. This opens up possibilities for cross-departmental workflows—marketing, operations, and finance all sharing insights effortlessly.</p>
  </li>
  <li>
    <p><strong>Cost Efficiency</strong>: By automating complex reasoning tasks, GPT-5.5 reduces the human overhead needed for data interpretation. This frees up teams to focus on innovation rather than grunt work.</p>
  </li>
</ol>

<hr />

<h2 id="practical-takeaway">Practical Takeaway</h2>

<p>If you’re leading engineering teams, start experimenting with GPT-5.5 in controlled environments. Databricks makes this easy by offering pre-built integrations for common enterprise use cases.</p>

<p>Here’s a quick TypeScript example to integrate GPT-5.5 into a serverless workflow:</p>

<div class="language-typescript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">import</span> <span class="p">{</span> <span class="nx">GPT55</span> <span class="p">}</span> <span class="k">from</span> <span class="dl">"</span><span class="s2">databricks-gpt</span><span class="dl">"</span><span class="p">;</span>  

<span class="kd">const</span> <span class="nx">agent</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">GPT55</span><span class="p">({</span> <span class="na">apiKey</span><span class="p">:</span> <span class="dl">"</span><span class="s2">your_api_key</span><span class="dl">"</span> <span class="p">});</span>  

<span class="kd">const</span> <span class="nx">query</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">Generate a weekly report from the attached customer feedback data.</span><span class="dl">"</span><span class="p">;</span>  
<span class="kd">const</span> <span class="nx">feedbackData</span> <span class="o">=</span> <span class="p">[</span>  
  <span class="p">{</span> <span class="na">customer</span><span class="p">:</span> <span class="dl">"</span><span class="s2">A</span><span class="dl">"</span><span class="p">,</span> <span class="na">feedback</span><span class="p">:</span> <span class="dl">"</span><span class="s2">Great product!</span><span class="dl">"</span><span class="p">,</span> <span class="na">sentiment</span><span class="p">:</span> <span class="dl">"</span><span class="s2">Positive</span><span class="dl">"</span> <span class="p">},</span>  
  <span class="p">{</span> <span class="na">customer</span><span class="p">:</span> <span class="dl">"</span><span class="s2">B</span><span class="dl">"</span><span class="p">,</span> <span class="na">feedback</span><span class="p">:</span> <span class="dl">"</span><span class="s2">Delivery was late.</span><span class="dl">"</span><span class="p">,</span> <span class="na">sentiment</span><span class="p">:</span> <span class="dl">"</span><span class="s2">Negative</span><span class="dl">"</span> <span class="p">},</span>  
<span class="p">];</span>  

<span class="nx">agent</span><span class="p">.</span><span class="nx">processMultimodal</span><span class="p">({</span> <span class="nx">query</span><span class="p">,</span> <span class="na">data</span><span class="p">:</span> <span class="nx">feedbackData</span> <span class="p">})</span>  
  <span class="p">.</span><span class="nx">then</span><span class="p">((</span><span class="nx">response</span><span class="p">)</span> <span class="o">=&gt;</span> <span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="dl">"</span><span class="s2">Generated Report:</span><span class="dl">"</span><span class="p">,</span> <span class="nx">response</span><span class="p">))</span>  
  <span class="p">.</span><span class="k">catch</span><span class="p">((</span><span class="nx">error</span><span class="p">)</span> <span class="o">=&gt;</span> <span class="nx">console</span><span class="p">.</span><span class="nx">error</span><span class="p">(</span><span class="dl">"</span><span class="s2">Error:</span><span class="dl">"</span><span class="p">,</span> <span class="nx">error</span><span class="p">));</span>  
</code></pre></div></div>

<p>This is where the future of enterprise workflows is heading—AI agents embedded into every layer of your systems, augmenting human capabilities and driving innovation.</p>

<p>So, what’s stopping you?</p>]]></content><author><name>Harry Zhao</name><email>harry.zhao19@gmail.com</email></author><category term="AI" /><category term="Engineering" /><summary type="html"><![CDATA[Databricks Brings GPT-5.5 to Enterprise Agent Workflows]]></summary></entry><entry><title type="html">Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model</title><link href="https://harry-zhao-au.github.io/blog/2026/05/13/show-hn-needle-we-distilled-gemini-tool-calling-into-a-26m-model/" rel="alternate" type="text/html" title="Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model" /><published>2026-05-13T00:00:00+00:00</published><updated>2026-05-13T00:00:00+00:00</updated><id>https://harry-zhao-au.github.io/blog/2026/05/13/show-hn-needle-we-distilled-gemini-tool-calling-into-a-26m-model</id><content type="html" xml:base="https://harry-zhao-au.github.io/blog/2026/05/13/show-hn-needle-we-distilled-gemini-tool-calling-into-a-26m-model/"><![CDATA[<h1 id="show-hn-needle-we-distilled-gemini-tool-calling-into-a-26m-model">Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model</h1>

<p>Can AI models be both powerful and lightweight? For years, the answer seemed to be “pick one.” But Needle just flipped the script. It’s a 26M parameter model that executes tool calling with surprising efficiency and accuracy, distilled from the Gemini architecture. If you’ve wrestled with bloated AI systems that need a fleet of GPUs to function, Needle feels like a breath of fresh air.</p>

<p>Here’s the kicker: Needle achieves this without sacrificing performance. It’s fast, precise, and small enough to run on devices you’d never consider for larger models. You can find the code <a href="https://github.com/cactus-compute/needle">here</a>. Let’s dig into why this matters, how it works, and what it means for the future of lightweight AI.</p>

<hr />

<h2 id="why-distill-gemini">Why Distill Gemini?</h2>

<p>Gemini tool calling models are impressive but hefty. They shine at orchestrating external tools — APIs, databases, or even automating workflows — but their size makes them hard to deploy at scale. Managing these models in production is like trying to ship a container full of lead: possible, but expensive and unwieldy.</p>

<p>Needle takes the essence of Gemini and squeezes it into a 26M parameter model. For comparison, GPT-3 has 175B parameters, and even “small” models like GPT-2 hover around 1.5B. Needle is orders of magnitude lighter. So why does size matter? Because lighter models unlock applications where computational resources are limited. Think IoT devices, edge servers, or even running directly on client-side devices like smartphones.</p>

<p>The distillation process boils down to capturing the decision-making abilities of the larger Gemini model while stripping away redundancy. It’s like compressing a high-resolution image: you retain the important details that matter for the task at hand without dragging around unnecessary pixels.</p>

<hr />

<h2 id="how-needle-achieves-efficiency">How Needle Achieves Efficiency</h2>

<p>Let’s get technical. Tool calling is about interpreting user input and triggering external systems — think SQL queries for databases, REST API calls for web services, or running shell commands. The model needs three things: comprehension, precision, and execution.</p>

<p>Needle achieves these through smart architectural design and training:</p>

<ol>
  <li>
    <p><strong>Focus on Intent Parsing</strong><br />
Needle is laser-focused on understanding the user’s intent. Instead of trying to generate verbose explanations or handle open-ended reasoning tasks, it specializes in mapping inputs directly to actionable commands. Think of it as a chatbot that skips the small talk and gets straight to business.</p>

    <p>Here’s a basic Python example using Needle for SQL generation:</p>

    <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">from</span> <span class="nn">needle</span> <span class="kn">import</span> <span class="n">ToolCaller</span>

<span class="n">model</span> <span class="o">=</span> <span class="n">ToolCaller</span><span class="p">.</span><span class="n">load_pretrained</span><span class="p">(</span><span class="s">"needle-26m"</span><span class="p">)</span>

<span class="n">user_input</span> <span class="o">=</span> <span class="s">"Find all customers who made purchases over $100 last month."</span>
<span class="n">tool_call</span> <span class="o">=</span> <span class="n">model</span><span class="p">.</span><span class="n">generate_tool_call</span><span class="p">(</span><span class="n">user_input</span><span class="p">)</span>

<span class="k">print</span><span class="p">(</span><span class="n">tool_call</span><span class="p">)</span>  
<span class="c1"># Output: {"tool": "sql_query", "query": "SELECT * FROM customers WHERE purchase_amount &gt; 100 AND purchase_date BETWEEN '2023-09-01' AND '2023-09-30';"}
</span></code></pre></div>    </div>

    <p>Notice how compact and actionable the output is? No fluff, just the exact query you need.</p>
  </li>
  <li>
    <p><strong>Optimized Attention Mechanisms</strong><br />
Needle doesn’t reinvent the transformer architecture but trims it down. Instead of sprawling attention layers that balloon memory usage, Needle prioritizes sparse attention patterns, focusing only on critical tokens. Think of it as reading the bolded sections of a report instead of the entire thing.</p>
  </li>
  <li>
    <p><strong>Knowledge Distillation</strong><br />
The training process uses Gemini as a teacher model. Needle learns by imitating Gemini’s outputs on a curated dataset of tool-calling tasks. This approach ensures that Needle inherits Gemini’s strengths without carrying its bulk. Distillation is not a new concept, but Needle shows how far it can be pushed.</p>
  </li>
</ol>

<hr />

<h2 id="implications-for-ai-augmented-systems">Implications for AI-Augmented Systems</h2>

<p>Needle isn’t just an academic curiosity — it’s a practical solution for real-world systems. Let’s look at some scenarios where it shines.</p>

<h3 id="1-banking-systems-fraud-detection-and-transaction-orchestration">1. <strong>Banking Systems: Fraud Detection and Transaction Orchestration</strong></h3>
<p>Banks often need models that can call multiple tools efficiently. For example, detecting fraudulent transactions might involve querying a database of transaction patterns, calling an external API for risk scoring, and triggering an alert system. With Needle’s lightweight design, these operations can happen on edge servers within a branch or even on customer devices for real-time checks.</p>

<p>Imagine a bank deploying Needle directly in its ATM network for dynamic fraud monitoring. Instead of relying on a centralized server, Needle runs locally, parsing transactions and instantly calling APIs for verification.</p>

<h3 id="2-logistics-platforms-dispatch-optimization">2. <strong>Logistics Platforms: Dispatch Optimization</strong></h3>
<p>In logistics, speed is everything. Dispatch platforms rely on AI to allocate drivers, optimize routes, and query supply chain databases. Large models can do this effectively but often require expensive cloud hosting. Needle flips the script — its small footprint allows logistics companies to deploy AI models closer to the action, like on regional servers or even vehicles.</p>

<p>Here’s a TypeScript mock-up of Needle generating API calls for dispatch:</p>

<div class="language-typescript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">import</span> <span class="p">{</span> <span class="nx">ToolCaller</span> <span class="p">}</span> <span class="k">from</span> <span class="dl">"</span><span class="s2">needle-ts</span><span class="dl">"</span><span class="p">;</span>

<span class="kd">const</span> <span class="nx">toolCaller</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">ToolCaller</span><span class="p">(</span><span class="dl">"</span><span class="s2">needle-26m</span><span class="dl">"</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">input</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">Find the nearest available driver to deliver package ID 12345.</span><span class="dl">"</span><span class="p">;</span>

<span class="kd">const</span> <span class="nx">toolCall</span> <span class="o">=</span> <span class="nx">toolCaller</span><span class="p">.</span><span class="nx">generateToolCall</span><span class="p">(</span><span class="nx">input</span><span class="p">);</span>

<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="nx">toolCall</span><span class="p">);</span>
<span class="c1">// Output: { tool: "dispatch_api", endpoint: "/drivers/nearest", payload: { package_id: 12345 } }</span>
</code></pre></div></div>

<p>This efficiency translates into faster decision-making, lower latency, and reduced infrastructure costs.</p>

<h3 id="3-distributed-platforms-cloud-native-services">3. <strong>Distributed Platforms: Cloud-Native Services</strong></h3>
<p>Needle aligns perfectly with the shift toward cloud-native architecture. Distributed systems thrive on lightweight components that can scale horizontally. Needle’s compact design means it can be deployed across hundreds of nodes without breaking the bank.</p>

<p>Imagine a Kubernetes cluster where each pod runs a Needle instance to interpret user commands, query APIs, and manage workflows. Scaling becomes trivial, and you avoid the headache of provisioning specialized GPU hardware.</p>

<hr />

<h2 id="the-future-of-lightweight-ai-models">The Future of Lightweight AI Models</h2>

<p>Needle isn’t just a one-off experiment — it’s part of a broader trend toward compact, efficient AI systems. Here’s what I see coming:</p>

<ul>
  <li>
    <p><strong>Edge AI Evolution:</strong> Models like Needle will redefine what’s possible on the edge. IoT devices, smart appliances, and autonomous systems all benefit from models that don’t need to phone home to a cloud server for processing.</p>
  </li>
  <li>
    <p><strong>Democratized AI:</strong> Small models lower the barrier to entry for startups and small businesses. You don’t need million-dollar infrastructure to deploy Needle — a modest server is enough. This levels the playing field in industries where AI was previously locked behind high costs.</p>
  </li>
  <li>
    <p><strong>Scaling Lightweight Models:</strong> Needle proves that small models can scale horizontally. Instead of one monolithic GPT-like instance, imagine thousands of Needles distributed across a platform, each handling specific tool-calling tasks.</p>
  </li>
</ul>

<hr />

<p>Needle isn’t just an incremental improvement — it’s a paradigm shift. By distilling tool-calling capabilities into a compact 26M model, it opens the door to AI applications that were previously infeasible. Whether you’re optimizing logistics, streamlining banking workflows, or building the next cloud-native platform, Needle makes it easier, faster, and cheaper.</p>

<p>Got questions? Dive into the <a href="https://github.com/cactus-compute/needle">GitHub repo</a> or let me know what you think. Let’s make lightweight AI the future of scalable systems.</p>]]></content><author><name>Harry Zhao</name><email>harry.zhao19@gmail.com</email></author><category term="AI" /><category term="Engineering" /><summary type="html"><![CDATA[Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model]]></summary></entry><entry><title type="html">Local AI needs to be the norm</title><link href="https://harry-zhao-au.github.io/blog/2026/05/11/local-ai-needs-to-be-the-norm/" rel="alternate" type="text/html" title="Local AI needs to be the norm" /><published>2026-05-11T00:00:00+00:00</published><updated>2026-05-11T00:00:00+00:00</updated><id>https://harry-zhao-au.github.io/blog/2026/05/11/local-ai-needs-to-be-the-norm</id><content type="html" xml:base="https://harry-zhao-au.github.io/blog/2026/05/11/local-ai-needs-to-be-the-norm/"><![CDATA[<h1 id="local-ai-needs-to-be-the-norm">Local AI needs to be the norm</h1>

<p>Let’s face it: AI is everywhere. But the way we deploy it is broken. For years, we’ve relied on cloud-based AI systems—train the model, deploy it to a server, make requests over the network, and call it a day. It works, but it’s far from ideal. Why? Because this model sacrifices privacy, speed, and reliability for convenience. Local AI—running models directly on devices—isn’t just some niche alternative. It’s the future, and frankly, it should already be the norm.</p>

<h2 id="why-local-ai-matters">Why local AI matters</h2>

<p>Here’s a thought experiment: Imagine you’re building an AI-powered fraud detection system for a bank. Every time a customer swipes their card, the transaction data gets shipped off to your cloud models, processed, and returned with a decision. What happens when the network goes down? Or latency spikes? Or worse, what if your cloud provider suffers a major outage? Suddenly, your system grinds to a halt. That’s unacceptable for something as critical as fraud detection.</p>

<p>Local AI solves these problems. By running models directly on the bank’s edge devices—ATMs, mobile apps, or even card terminals—you eliminate the dependency on the cloud. No network? No problem. Decisions happen locally, instantly.</p>

<p>And privacy? It’s a game-changer. Sensitive customer data never leaves the device, sidestepping compliance headaches like GDPR, HIPAA, or Australia’s Privacy Act. In a world where data breaches are daily news, keeping data on-device isn’t just smart—it’s mandatory.</p>

<h2 id="the-technical-challenges">The technical challenges</h2>

<p>Of course, local AI isn’t without hurdles. Deploying models locally isn’t as simple as dragging and dropping your TensorFlow code onto a smartphone. You’ve got several hard problems to solve:</p>

<h3 id="1-model-size-and-optimization">1. Model size and optimization</h3>

<p>Most AI models are too big for local devices. Take GPT-3, for instance—it has 175 billion parameters. Good luck running that on your phone without melting the CPU. Local AI demands models that are lean, efficient, and tailored for edge hardware.</p>

<p>Techniques like quantization, pruning, and distillation come to the rescue here. Quantization reduces the precision of model weights (e.g., from 32-bit floats to 8-bit integers). Pruning removes redundant parts of the model. Distillation trains smaller models to mimic larger ones. Together, these methods shrink models while retaining accuracy.</p>

<p>Want to see it in action? Here’s a Python example using TensorFlow Lite:</p>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">tensorflow</span> <span class="k">as</span> <span class="n">tf</span>
<span class="kn">import</span> <span class="nn">tensorflow_model_optimization</span> <span class="k">as</span> <span class="n">tfmot</span>

<span class="c1"># Load your model
</span><span class="n">model</span> <span class="o">=</span> <span class="n">tf</span><span class="p">.</span><span class="n">keras</span><span class="p">.</span><span class="n">models</span><span class="p">.</span><span class="n">load_model</span><span class="p">(</span><span class="s">"my_model.h5"</span><span class="p">)</span>

<span class="c1"># Apply quantization
</span><span class="n">quantized_model</span> <span class="o">=</span> <span class="n">tfmot</span><span class="p">.</span><span class="n">quantization</span><span class="p">.</span><span class="n">keras</span><span class="p">.</span><span class="n">quantize_model</span><span class="p">(</span><span class="n">model</span><span class="p">)</span>

<span class="c1"># Save the optimized model
</span><span class="n">quantized_model</span><span class="p">.</span><span class="n">save</span><span class="p">(</span><span class="s">"my_model_quantized.tflite"</span><span class="p">)</span>
</code></pre></div></div>

<p>With this, you can run your model on a Raspberry Pi or even a low-powered IoT device. No cloud required.</p>

<h3 id="2-hardware-compatibility">2. Hardware compatibility</h3>

<p>Not all devices are created equal. Running AI locally means dealing with a fragmented hardware ecosystem—phones, laptops, smart cameras, industrial sensors, you name it. Each has its quirks: ARM vs x86, GPU vs TPU, varying amounts of RAM and storage.</p>

<p>Frameworks like ONNX (Open Neural Network Exchange) help bridge the gap. ONNX lets you export models in a universal format and run them on different devices with minimal changes. Here’s how it works in Python:</p>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">onnx</span>
<span class="kn">from</span> <span class="nn">keras2onnx</span> <span class="kn">import</span> <span class="n">convert_keras</span>

<span class="c1"># Convert Keras model to ONNX
</span><span class="n">model</span> <span class="o">=</span> <span class="n">tf</span><span class="p">.</span><span class="n">keras</span><span class="p">.</span><span class="n">models</span><span class="p">.</span><span class="n">load_model</span><span class="p">(</span><span class="s">"my_model.h5"</span><span class="p">)</span>
<span class="n">onnx_model</span> <span class="o">=</span> <span class="n">convert_keras</span><span class="p">(</span><span class="n">model</span><span class="p">,</span> <span class="s">"my_model.onnx"</span><span class="p">)</span>

<span class="c1"># Save the ONNX model
</span><span class="n">onnx</span><span class="p">.</span><span class="n">save_model</span><span class="p">(</span><span class="n">onnx_model</span><span class="p">,</span> <span class="s">"my_model.onnx"</span><span class="p">)</span>
</code></pre></div></div>

<p>Once exported, you can deploy the model anywhere, from your laptop to an IoT sensor running ONNX Runtime.</p>

<h3 id="3-continuous-updates">3. Continuous updates</h3>

<p>AI models aren’t static—they need updates to stay relevant. Fraud patterns evolve, customer preferences shift, and hardware capabilities improve. Local deployment complicates updates since you can’t just swap a model in the cloud. Instead, you need robust mechanisms for distributing updates to devices.</p>

<p>One solution is to use delta updates. Instead of shipping the entire model, you only send the “diff”—the parts of the model that changed. This minimizes bandwidth usage and speeds up the update process. In TypeScript, you might use a library like <code class="language-plaintext highlighter-rouge">bsdiff-node</code> to generate binary diffs:</p>

<div class="language-typescript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">const</span> <span class="nx">bsdiff</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">bsdiff-node</span><span class="dl">'</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">fs</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">fs</span><span class="dl">'</span><span class="p">);</span>

<span class="c1">// Generate diff between old and new models</span>
<span class="kd">const</span> <span class="nx">oldModel</span> <span class="o">=</span> <span class="nx">fs</span><span class="p">.</span><span class="nx">readFileSync</span><span class="p">(</span><span class="dl">"</span><span class="s2">model_v1.bin</span><span class="dl">"</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">newModel</span> <span class="o">=</span> <span class="nx">fs</span><span class="p">.</span><span class="nx">readFileSync</span><span class="p">(</span><span class="dl">"</span><span class="s2">model_v2.bin</span><span class="dl">"</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">diff</span> <span class="o">=</span> <span class="nx">bsdiff</span><span class="p">.</span><span class="nx">diff</span><span class="p">(</span><span class="nx">oldModel</span><span class="p">,</span> <span class="nx">newModel</span><span class="p">);</span>

<span class="c1">// Save the diff file</span>
<span class="nx">fs</span><span class="p">.</span><span class="nx">writeFileSync</span><span class="p">(</span><span class="dl">"</span><span class="s2">model_update.diff</span><span class="dl">"</span><span class="p">,</span> <span class="nx">diff</span><span class="p">);</span>
</code></pre></div></div>

<p>Devices apply the diff locally to reconstruct the updated model.</p>

<h2 id="the-advantages">The advantages</h2>

<p>Local AI isn’t just about solving technical problems—it unlocks entirely new possibilities.</p>

<h3 id="privacy-by-default">Privacy by default</h3>

<p>Banks, hospitals, and logistics companies deal with sensitive data that can’t leave the premises. Local AI ensures compliance without the need for elaborate encryption schemes or third-party audits. The data stays where it belongs: on your devices.</p>

<h3 id="low-latency">Low latency</h3>

<p>When milliseconds matter—like detecting fraud during a transaction or guiding a drone around obstacles—local AI is unbeatable. There’s no round-trip to the cloud, no waiting for a response. Decisions happen in real time.</p>

<h3 id="reliability-in-distributed-systems">Reliability in distributed systems</h3>

<p>Cloud outages are rare but catastrophic when they happen. Remember the AWS outage that took down half the internet? With local AI, you’re insulated from these risks. Distributed systems become truly distributed, with each node capable of functioning independently.</p>

<h2 id="the-future-of-edge-computing">The future of edge computing</h2>

<p>Local AI is already reshaping edge computing. Devices like NVIDIA Jetson, Apple’s Neural Engine, and Google’s Coral TPU are tailor-made for running AI models locally. And decentralized platforms like federated learning take it a step further, enabling devices to collaboratively train models without sharing data.</p>

<p>Imagine a fleet of delivery drones dynamically optimizing routes based on traffic and weather, all without relying on a central server. Or wearable health monitors detecting anomalies and alerting users instantly, without sending private medical data to the cloud. These aren’t science fiction—they’re the logical next step in AI deployment.</p>

<h2 id="practical-takeaway">Practical takeaway</h2>

<p>If you’re building AI systems today, ask yourself: <em>Does this really need the cloud?</em> Start experimenting with local AI. Optimize your models. Test them on edge devices. The tools are there—TensorFlow Lite, ONNX, PyTorch Mobile, Core ML, you name it. It’s not just about staying ahead of the curve. It’s about building systems that are faster, safer, and more resilient.</p>

<p>The age of local AI is here. Let’s make it the norm.</p>]]></content><author><name>Harry Zhao</name><email>harry.zhao19@gmail.com</email></author><category term="AI" /><category term="Engineering" /><summary type="html"><![CDATA[Local AI needs to be the norm]]></summary></entry><entry><title type="html">Agents need control flow, not more prompts</title><link href="https://harry-zhao-au.github.io/blog/2026/05/08/agents-need-control-flow-not-more-prompts/" rel="alternate" type="text/html" title="Agents need control flow, not more prompts" /><published>2026-05-08T00:00:00+00:00</published><updated>2026-05-08T00:00:00+00:00</updated><id>https://harry-zhao-au.github.io/blog/2026/05/08/agents-need-control-flow-not-more-prompts</id><content type="html" xml:base="https://harry-zhao-au.github.io/blog/2026/05/08/agents-need-control-flow-not-more-prompts/"><![CDATA[<h1 id="agents-need-control-flow-not-more-prompts">Agents Need Control Flow, Not More Prompts</h1>

<p>AI agents are getting smarter. We’ve seen mind-blowing advances in large language models (LLMs), reinforcement learning, and distributed systems. But here’s the problem: when these agents are tasked with solving complex, multi-step workflows, engineers are increasingly relying on prompt engineering tricks instead of addressing the real bottleneck—control flow.</p>

<p>Let me be blunt: prompts are great for one-off tasks. But they fall apart when you need deterministic, repeatable, and scalable workflows. What AI agents really need is robust control flow. Not more convoluted prompts. Not more layers of abstraction. Just solid, predictable mechanisms to handle decisions, loops, and branching logic.</p>

<p>Why does this matter? Because without control flow, AI systems are brittle. They fail unpredictably. And when they fail, debugging is a nightmare. If you’re building anything mission-critical—say, fraud detection in banking or supply chain optimization in logistics—you can’t afford that.</p>

<p>Let’s dig deeper.</p>

<hr />

<h2 id="the-problem-with-prompt-driven-architectures">The Problem with Prompt-Driven Architectures</h2>

<p>Prompt engineering has become the go-to solution for building AI agents. It’s easy to see why: you write a clever prompt, toss it into an LLM, and voilà—your agent spits out reasonable outputs. But prompts are inherently fuzzy. They’re probabilistic by nature, and that’s fine for generating text or answering trivia.</p>

<p>Now imagine asking an AI agent to coordinate a fleet of delivery trucks. The agent needs to:</p>

<ol>
  <li>Parse incoming orders.</li>
  <li>Allocate trucks based on capacity.</li>
  <li>Optimize routes to minimize fuel costs.</li>
  <li>Monitor real-time traffic for rerouting.</li>
</ol>

<p>Sure, you could write a massive prompt describing all of this in excruciating detail. But as soon as something changes—say, a truck breaks down—you’re back to square one, rewriting prompts and hoping the agent “gets it.” This approach doesn’t scale. It’s like duct-taping a leaky pipe instead of replacing it.</p>

<p>What’s missing here? Deterministic control flow.</p>

<hr />

<h2 id="why-control-flow-is-critical-for-ai-agents">Why Control Flow is Critical for AI Agents</h2>

<p>Control flow is the backbone of traditional software engineering. If-else statements, loops, error handling, function calls—these are the tools we use to design systems that work reliably, even in edge cases. AI agents need the same principles.</p>

<p>Take banking systems, for example. Banks use automated workflows to approve loans. A typical flow might look like:</p>

<ol>
  <li>Check the applicant’s credit score.</li>
  <li>Verify income documents.</li>
  <li>Calculate debt-to-income ratio.</li>
  <li>Decide: approve or reject.</li>
</ol>

<p>This isn’t rocket science. It’s a simple sequence of operations with clear branching logic. If the credit score is below 600, reject. If income documents are missing, flag for manual review.</p>

<p>Now imagine implementing this with prompt-driven AI. You’d need to craft prompts for every possible scenario, hoping the agent doesn’t hallucinate or miss critical steps. It’s a recipe for chaos.</p>

<p>With control flow, you can enforce structure. You can guarantee that every decision follows a deterministic path. No guesswork. No hand-holding.</p>

<hr />

<h2 id="how-control-flow-enhances-distributed-ai-systems">How Control Flow Enhances Distributed AI Systems</h2>

<p>Distributed AI systems—think fleets of agents working together—are particularly vulnerable to the lack of control flow. Why? Because these systems depend on coordination. If one agent makes a bad decision, it can cascade through the entire network.</p>

<p>Let’s use logistics as an example. Imagine a system where agents coordinate deliveries across 10 warehouses. Each agent needs to:</p>

<ol>
  <li>Process incoming orders.</li>
  <li>Communicate inventory levels to other agents.</li>
  <li>Decide whether to ship locally or transfer stock between warehouses.</li>
</ol>

<p>Without control flow, these agents are essentially guessing. One agent might decide to transfer stock without checking inventory levels. Another might prioritize low-value orders over high-value ones because the prompt wasn’t specific enough. These errors compound, and suddenly your entire supply chain is out of sync.</p>

<p>Control flow solves this. You can implement guardrails—rules that every agent must follow. For example:</p>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">process_order</span><span class="p">(</span><span class="n">order</span><span class="p">):</span>
    <span class="k">if</span> <span class="ow">not</span> <span class="n">is_valid_order</span><span class="p">(</span><span class="n">order</span><span class="p">):</span>
        <span class="k">return</span> <span class="s">"Reject: Invalid order"</span>
    
    <span class="k">if</span> <span class="n">is_high_priority</span><span class="p">(</span><span class="n">order</span><span class="p">):</span>
        <span class="n">allocate_trucks</span><span class="p">(</span><span class="n">order</span><span class="p">)</span>
    <span class="k">else</span><span class="p">:</span>
        <span class="n">queue_for_batch_processing</span><span class="p">(</span><span class="n">order</span><span class="p">)</span>

<span class="k">def</span> <span class="nf">allocate_trucks</span><span class="p">(</span><span class="n">order</span><span class="p">):</span>
    <span class="n">available_trucks</span> <span class="o">=</span> <span class="n">get_available_trucks</span><span class="p">()</span>
    <span class="k">if</span> <span class="ow">not</span> <span class="n">available_trucks</span><span class="p">:</span>
        <span class="k">return</span> <span class="s">"Error: No trucks available"</span>
    
    <span class="n">optimize_routes</span><span class="p">(</span><span class="n">available_trucks</span><span class="p">,</span> <span class="n">order</span><span class="p">)</span>
</code></pre></div></div>

<p>With code like this, every decision is explicit. Every error has a fallback. Distributed agents can communicate reliably, knowing that their peers are following the same deterministic rules.</p>

<hr />

<h2 id="practical-implementations-combining-ai-and-control-flow">Practical Implementations: Combining AI and Control Flow</h2>

<p>So how do you actually implement control flow in AI systems? The answer lies in hybrid architectures. Pair your AI agent with a traditional programming framework. Let the agent handle high-level reasoning, but keep control flow grounded in deterministic code.</p>

<p>Here’s a practical example in a TypeScript-based customer service chatbot:</p>

<div class="language-typescript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">function</span> <span class="nx">handleCustomerQuery</span><span class="p">(</span><span class="nx">query</span><span class="p">:</span> <span class="kr">string</span><span class="p">):</span> <span class="kr">string</span> <span class="p">{</span>
    <span class="kd">const</span> <span class="nx">intent</span> <span class="o">=</span> <span class="nx">classifyIntent</span><span class="p">(</span><span class="nx">query</span><span class="p">);</span> <span class="c1">// AI-based intent classification</span>
    
    <span class="k">switch</span> <span class="p">(</span><span class="nx">intent</span><span class="p">)</span> <span class="p">{</span>
        <span class="k">case</span> <span class="dl">"</span><span class="s2">billing</span><span class="dl">"</span><span class="p">:</span>
            <span class="k">return</span> <span class="nx">handleBillingQuery</span><span class="p">(</span><span class="nx">query</span><span class="p">);</span>
        <span class="k">case</span> <span class="dl">"</span><span class="s2">technical_support</span><span class="dl">"</span><span class="p">:</span>
            <span class="k">return</span> <span class="nx">handleTechSupport</span><span class="p">(</span><span class="nx">query</span><span class="p">);</span>
        <span class="nl">default</span><span class="p">:</span>
            <span class="k">return</span> <span class="dl">"</span><span class="s2">Sorry, I didn't understand your query.</span><span class="dl">"</span><span class="p">;</span>
    <span class="p">}</span>
<span class="p">}</span>

<span class="kd">function</span> <span class="nx">classifyIntent</span><span class="p">(</span><span class="nx">query</span><span class="p">:</span> <span class="kr">string</span><span class="p">):</span> <span class="kr">string</span> <span class="p">{</span>
    <span class="c1">// Use an AI model to classify the intent</span>
    <span class="k">return</span> <span class="nx">aiModel</span><span class="p">.</span><span class="nx">predict</span><span class="p">(</span><span class="nx">query</span><span class="p">);</span>
<span class="p">}</span>

<span class="kd">function</span> <span class="nx">handleBillingQuery</span><span class="p">(</span><span class="nx">query</span><span class="p">:</span> <span class="kr">string</span><span class="p">):</span> <span class="kr">string</span> <span class="p">{</span>
    <span class="k">if</span> <span class="p">(</span><span class="nx">query</span><span class="p">.</span><span class="nx">includes</span><span class="p">(</span><span class="dl">"</span><span class="s2">refund</span><span class="dl">"</span><span class="p">))</span> <span class="p">{</span>
        <span class="k">return</span> <span class="dl">"</span><span class="s2">Please provide your order number to process the refund.</span><span class="dl">"</span><span class="p">;</span>
    <span class="p">}</span>
    <span class="k">return</span> <span class="dl">"</span><span class="s2">For billing inquiries, visit our support page.</span><span class="dl">"</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div></div>

<p>Notice the separation of roles here. The AI handles intent classification—a fuzzy task where probabilistic reasoning makes sense. But the actual workflow is deterministic. You can debug it. You can test it. And you can be confident that it’ll work the same way tomorrow as it does today.</p>

<hr />

<h2 id="the-takeaway">The Takeaway</h2>

<p>AI agents don’t need more prompts. They need structure. Deterministic control flow is the key to building systems that are reliable, scalable, and easy to debug. Whether you’re working in banking, logistics, or customer service, the principle is the same: use AI for reasoning, but enforce control flow with code.</p>

<p>If you’re still relying on prompt engineering hacks to build complex workflows, it’s time to rethink your approach. Control flow isn’t just a nice-to-have—it’s a necessity. The sooner we embrace it, the sooner we can build AI systems that actually work.</p>]]></content><author><name>Harry Zhao</name><email>harry.zhao19@gmail.com</email></author><category term="AI" /><category term="Engineering" /><summary type="html"><![CDATA[Agents Need Control Flow, Not More Prompts]]></summary></entry><entry><title type="html">OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors</title><link href="https://harry-zhao-au.github.io/blog/2026/05/03/openai-s-o1-correctly-diagnosed-67-of-er-patients-vs-50-55-by-triage-doctors/" rel="alternate" type="text/html" title="OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors" /><published>2026-05-03T00:00:00+00:00</published><updated>2026-05-03T00:00:00+00:00</updated><id>https://harry-zhao-au.github.io/blog/2026/05/03/openai-s-o1-correctly-diagnosed-67-of-er-patients-vs-50-55-by-triage-doctors</id><content type="html" xml:base="https://harry-zhao-au.github.io/blog/2026/05/03/openai-s-o1-correctly-diagnosed-67-of-er-patients-vs-50-55-by-triage-doctors/"><![CDATA[<h1 id="ai-diagnosing-er-patients-better-than-doctors-openais-o1-changes-the-game">AI Diagnosing ER Patients Better Than Doctors? OpenAI’s o1 Changes the Game</h1>

<p>Imagine walking into an emergency room with chest pain, shortness of breath, and a nagging fear of something serious. The triage doctor assesses you, makes a preliminary diagnosis, and sets your care path. But what if an AI could do that better? Not just faster, but more accurately?</p>

<p>That’s exactly what OpenAI’s “o1” has demonstrated in a Harvard-led trial. It correctly diagnosed 67% of ER patients compared to 50–55% by triage doctors. For a field where every decision could mean life or death, that’s massive.</p>

<p>But this isn’t just about bragging rights for AI developers. It’s about rethinking emergency care workflows, addressing scalability in healthcare, and handling ethical landmines along the way. Let’s dive into how OpenAI’s o1 works, why it matters, and what’s next.</p>

<hr />

<h2 id="how-ai-is-changing-emergency-care">How AI Is Changing Emergency Care</h2>

<p>Emergency rooms are chaos. Patients come in with vague symptoms, incomplete histories, and often in critical condition. Doctors rely on experience, pattern recognition, and limited time to make high-stakes decisions.</p>

<p>AI doesn’t get overwhelmed. It doesn’t experience cognitive fatigue after a 12-hour shift. Systems like OpenAI’s o1 ingest patient data — symptoms, vitals, medical history — and output diagnoses with probabilities.</p>

<p>Here’s what’s groundbreaking: It’s not just about raw accuracy. AI augments human decision-making. In the Harvard study, triage doctors paired with o1 improved their diagnostic rates to 75%. That’s collaboration, not replacement.</p>

<p>For example, consider a patient presenting with abdominal pain. Is it appendicitis? A ruptured ovarian cyst? Or just bad dinner? Triage doctors often lean on heuristics and personal experience. o1, on the other hand, processes the patient’s history, lab results, and even similar anonymized cases from distributed datasets to suggest diagnoses ranked by likelihood.</p>

<hr />

<h2 id="the-technical-architecture-behind-o1">The Technical Architecture Behind o1</h2>

<p>Let’s get nerdy for a second. What makes o1 tick?</p>

<p>At its core, o1 relies on a multi-modal transformer. Unlike traditional NLP models that focus solely on text, multi-modal transformers integrate diverse data types:</p>

<ul>
  <li><strong>Text input</strong>: Chief complaints, descriptions of symptoms, and physician notes.</li>
  <li><strong>Numerical data</strong>: Vitals like heart rate, blood pressure, and oxygen saturation.</li>
  <li><strong>Imaging</strong>: X-rays, CT scans, and ultrasounds processed through convolutional layers before feeding into the transformer.</li>
</ul>

<p>Here’s a possible high-level architecture:</p>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">torch</span>  
<span class="kn">from</span> <span class="nn">transformers</span> <span class="kn">import</span> <span class="n">BertModel</span>  
<span class="kn">from</span> <span class="nn">torchvision.models</span> <span class="kn">import</span> <span class="n">resnet50</span>  

<span class="k">class</span> <span class="nc">O1EmergencyModel</span><span class="p">(</span><span class="n">torch</span><span class="p">.</span><span class="n">nn</span><span class="p">.</span><span class="n">Module</span><span class="p">):</span>  
    <span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>  
        <span class="nb">super</span><span class="p">().</span><span class="n">__init__</span><span class="p">()</span>  
        <span class="bp">self</span><span class="p">.</span><span class="n">text_model</span> <span class="o">=</span> <span class="n">BertModel</span><span class="p">.</span><span class="n">from_pretrained</span><span class="p">(</span><span class="s">"bert-base-uncased"</span><span class="p">)</span>  
        <span class="bp">self</span><span class="p">.</span><span class="n">image_model</span> <span class="o">=</span> <span class="n">resnet50</span><span class="p">(</span><span class="n">pretrained</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>  
        <span class="bp">self</span><span class="p">.</span><span class="n">fc_text</span> <span class="o">=</span> <span class="n">torch</span><span class="p">.</span><span class="n">nn</span><span class="p">.</span><span class="n">Linear</span><span class="p">(</span><span class="mi">768</span><span class="p">,</span> <span class="mi">256</span><span class="p">)</span>  <span class="c1"># BERT text embeddings  
</span>        <span class="bp">self</span><span class="p">.</span><span class="n">fc_image</span> <span class="o">=</span> <span class="n">torch</span><span class="p">.</span><span class="n">nn</span><span class="p">.</span><span class="n">Linear</span><span class="p">(</span><span class="mi">1000</span><span class="p">,</span> <span class="mi">256</span><span class="p">)</span>  <span class="c1"># ResNet image features  
</span>        <span class="bp">self</span><span class="p">.</span><span class="n">fc_combined</span> <span class="o">=</span> <span class="n">torch</span><span class="p">.</span><span class="n">nn</span><span class="p">.</span><span class="n">Linear</span><span class="p">(</span><span class="mi">512</span><span class="p">,</span> <span class="mi">3</span><span class="p">)</span>  <span class="c1"># Predict top 3 diagnoses  
</span>
    <span class="k">def</span> <span class="nf">forward</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">text_inputs</span><span class="p">,</span> <span class="n">image_inputs</span><span class="p">):</span>  
        <span class="n">text_features</span> <span class="o">=</span> <span class="bp">self</span><span class="p">.</span><span class="n">text_model</span><span class="p">(</span><span class="o">**</span><span class="n">text_inputs</span><span class="p">).</span><span class="n">pooler_output</span>  
        <span class="n">text_features</span> <span class="o">=</span> <span class="bp">self</span><span class="p">.</span><span class="n">fc_text</span><span class="p">(</span><span class="n">text_features</span><span class="p">)</span>  

        <span class="n">image_features</span> <span class="o">=</span> <span class="bp">self</span><span class="p">.</span><span class="n">image_model</span><span class="p">(</span><span class="n">image_inputs</span><span class="p">)</span>  
        <span class="n">image_features</span> <span class="o">=</span> <span class="bp">self</span><span class="p">.</span><span class="n">fc_image</span><span class="p">(</span><span class="n">image_features</span><span class="p">)</span>  

        <span class="n">combined</span> <span class="o">=</span> <span class="n">torch</span><span class="p">.</span><span class="n">cat</span><span class="p">((</span><span class="n">text_features</span><span class="p">,</span> <span class="n">image_features</span><span class="p">),</span> <span class="n">dim</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span>  
        <span class="k">return</span> <span class="bp">self</span><span class="p">.</span><span class="n">fc_combined</span><span class="p">(</span><span class="n">combined</span><span class="p">)</span>  

<span class="c1"># Example usage  
# text_inputs = {"input_ids": ..., "attention_mask": ...}  
# image_inputs = torch.randn(1, 3, 224, 224)  # Dummy image tensor  
# model = O1EmergencyModel()  
# predictions = model(text_inputs, image_inputs)  
</span></code></pre></div></div>

<p>This multi-modal approach lets o1 synthesize complex datasets, learn contextual relationships, and adapt to new cases. The implications for distributed healthcare platforms are staggering.</p>

<p>Imagine rural clinics feeding patient data into a cloud-hosted o1 instance. They get back ranked diagnoses and treatment recommendations — all without needing a specialist on-site. Distributed AI systems like this could democratize access to high-quality care.</p>

<hr />

<h2 id="the-ethical-and-regulatory-minefield">The Ethical and Regulatory Minefield</h2>

<p>But let’s not get ahead of ourselves. Deploying AI in healthcare isn’t just a technical challenge; it’s a moral and legal one.</p>

<p>First, <strong>accountability</strong>. If o1 suggests a misdiagnosis, who’s responsible? The doctor? OpenAI? The hospital? In high-stakes scenarios like emergency medicine, errors aren’t just costly — they’re catastrophic. Regulators need to figure out liability frameworks before widespread adoption.</p>

<p>Second, <strong>bias in training data</strong>. AI systems are only as good as their datasets. If o1 was trained predominantly on data from urban hospitals serving affluent populations, how well does it handle cases from underserved rural areas? Training data must be representative and diverse.</p>

<p>Third, <strong>trust and transparency</strong>. Will patients trust diagnoses from an AI? Many people have reservations about “machines making life-or-death decisions.” Transparency is crucial here. Patients — and physicians — need to understand how o1 arrives at its recommendations.</p>

<p>Finally, <strong>integration with healthcare workflows</strong>. ERs are already stretched thin. Adding an AI system can’t mean adding complexity. Instead, it needs to integrate seamlessly into existing processes. That’s a software engineering problem as much as a medical one.</p>

<hr />

<h2 id="why-this-matters-now">Why This Matters Now</h2>

<p>Here’s the big picture: Healthcare systems worldwide are under immense pressure. Aging populations, staffing shortages, and rising costs are straining resources. AI isn’t a panacea, but it’s a powerful tool to address these challenges.</p>

<p>The timing couldn’t be better. AI models like OpenAI’s o1 prove that machines can surpass human performance in diagnostics. But more importantly, they show that humans and machines together are even better.</p>

<p>If you’re a software engineer wondering how to contribute, focus on building systems that integrate AI into real-world workflows. Think usability, interoperability, and reliability.</p>

<p>If you’re a healthcare administrator, start exploring pilot programs for AI adoption. The Harvard study is a wake-up call. The longer we wait, the more lives we risk.</p>

<p>And if you’re a patient? Don’t fear AI in medicine. It’s not here to replace your doctor. It’s here to help them save your life.</p>

<p>Every day we delay integrating systems like o1, we lose opportunities to deliver better care. So, what’s stopping us?</p>

<hr />

<p>Sources:</p>
<ul>
  <li><a href="https://www.theguardian.com/technology/2026/apr/30/ai-outperforms-doctors-in-harvard-trial-of-emergency-triage-diagnoses">The Guardian on AI outperforming doctors in Harvard trials</a></li>
  <li><a href="https://deepmind.google/blog/ai-co-clinician/">Google DeepMind Blog on AI as co-clinician</a></li>
</ul>]]></content><author><name>Harry Zhao</name><email>harry.zhao19@gmail.com</email></author><category term="AI" /><category term="Engineering" /><summary type="html"><![CDATA[AI Diagnosing ER Patients Better Than Doctors? OpenAI’s o1 Changes the Game]]></summary></entry><entry><title type="html">Decoupled DiLoCo: A new frontier for resilient, distributed AI training</title><link href="https://harry-zhao-au.github.io/blog/2026/04/29/decoupled-diloco-a-new-frontier-for-resilient-distributed-ai-training/" rel="alternate" type="text/html" title="Decoupled DiLoCo: A new frontier for resilient, distributed AI training" /><published>2026-04-29T00:00:00+00:00</published><updated>2026-04-29T00:00:00+00:00</updated><id>https://harry-zhao-au.github.io/blog/2026/04/29/decoupled-diloco-a-new-frontier-for-resilient-distributed-ai-training</id><content type="html" xml:base="https://harry-zhao-au.github.io/blog/2026/04/29/decoupled-diloco-a-new-frontier-for-resilient-distributed-ai-training/"><![CDATA[<h1 id="decoupled-diloco-a-new-frontier-for-resilient-distributed-ai-training">Decoupled DiLoCo: A New Frontier for Resilient, Distributed AI Training</h1>

<p>Distributed AI training is hard. Anyone who’s worked on scaling AI workflows across multiple nodes knows this. You start with optimism—throw more hardware at the problem, and you’ll get faster results. But then, the cracks appear. One node goes down, the whole workflow stalls. Communication bottlenecks. Synchronization overhead. The list of headaches is endless.</p>

<p>Enter Decoupled DiLoCo. If you haven’t heard about it yet, Google DeepMind recently introduced this new framework for distributed AI training, and it’s nothing short of game-changing. It’s built for resilience, scalability, and efficiency. I’ve been diving into its technical details and thinking about how it applies to real-world enterprise problems, especially in industries like banking and logistics. Spoiler: this isn’t just another buzzword. Decoupled DiLoCo has practical applications today, and it’s worth paying attention to.</p>

<hr />

<h2 id="what-is-decoupled-diloco">What Is Decoupled DiLoCo?</h2>

<p>Let’s break it down. “DiLoCo” stands for <strong>Distributed Local Coordination</strong>, a method for parallelizing AI training across multiple nodes. Traditional distributed training frameworks rely on centralized coordination—e.g., a parameter server or master node that synchronizes all workers. This approach makes systems fragile. If the master node fails, everything grinds to a halt.</p>

<p>DiLoCo solves this fragility by decentralizing coordination. Instead of one master node, each worker node operates independently and locally coordinates with its neighbors. The “Decoupled” part of Decoupled DiLoCo takes this one step further: it removes tight coupling between nodes entirely. Workers can drop in and out without disrupting the training process. It’s like moving from a rigid orchestra to a jazz band—improvisation replaces strict synchronization.</p>

<hr />

<h2 id="why-does-this-matter">Why Does This Matter?</h2>

<h3 id="resilience">Resilience</h3>

<p>Here’s a scenario: You’re running a real-time fraud detection system for a global bank. The AI model needs to be retrained every hour using fresh transaction data. Your distributed training framework spans nodes across multiple cloud regions. Suddenly, a network outage takes down nodes in Europe. With traditional centralized coordination, this outage would derail the entire training job. With Decoupled DiLoCo, the remaining nodes continue training without missing a beat. The system is resilient by design.</p>

<h3 id="scalability">Scalability</h3>

<p>Scalability isn’t just about adding more nodes—it’s about doing so without diminishing returns. In cloud-native platforms, like AWS or Azure, resource allocation fluctuates. Decoupled DiLoCo thrives in this environment. Because coordination is local and decoupled, adding or removing nodes dynamically doesn’t introduce overhead. Need to scale your training job from 10 nodes to 100? Go ahead. Decoupled DiLoCo handles it seamlessly.</p>

<h3 id="efficiency">Efficiency</h3>

<p>Efficiency is where Decoupled DiLoCo shines. By decentralizing coordination, the framework reduces communication overhead. Workers exchange updates only with their immediate neighbors, not the entire cluster. This local-first approach minimizes latency and boosts throughput. For large-scale models, like GPT-style transformers, these savings translate to faster training times and lower compute costs.</p>

<hr />

<h2 id="how-does-it-work-a-technical-dive">How Does It Work? A Technical Dive</h2>

<p>Let’s get into the weeds. In Decoupled DiLoCo, each worker follows three key steps:</p>

<ol>
  <li><strong>Local Training</strong>: Workers independently process their data shards and compute gradients.</li>
  <li><strong>Neighbor Exchange</strong>: Workers share updates with their immediate neighbors, using a lightweight protocol for synchronization.</li>
  <li><strong>Global Aggregation</strong>: Periodically, all workers contribute to a global model update (optional, depending on the use case).</li>
</ol>

<p>Here’s an example of how this might look in code. Assume we’re implementing Decoupled DiLoCo in Python using PyTorch:</p>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">torch</span>
<span class="kn">from</span> <span class="nn">torch.distributed</span> <span class="kn">import</span> <span class="n">init_process_group</span><span class="p">,</span> <span class="n">all_reduce</span>

<span class="c1"># Initialize distributed process group
</span><span class="n">init_process_group</span><span class="p">(</span><span class="n">backend</span><span class="o">=</span><span class="s">"nccl"</span><span class="p">)</span>

<span class="c1"># Local training loop
</span><span class="k">def</span> <span class="nf">local_training_step</span><span class="p">(</span><span class="n">model</span><span class="p">,</span> <span class="n">data_loader</span><span class="p">,</span> <span class="n">optimizer</span><span class="p">):</span>
    <span class="k">for</span> <span class="n">data</span><span class="p">,</span> <span class="n">target</span> <span class="ow">in</span> <span class="n">data_loader</span><span class="p">:</span>
        <span class="n">optimizer</span><span class="p">.</span><span class="n">zero_grad</span><span class="p">()</span>
        <span class="n">output</span> <span class="o">=</span> <span class="n">model</span><span class="p">(</span><span class="n">data</span><span class="p">)</span>
        <span class="n">loss</span> <span class="o">=</span> <span class="n">torch</span><span class="p">.</span><span class="n">nn</span><span class="p">.</span><span class="n">functional</span><span class="p">.</span><span class="n">cross_entropy</span><span class="p">(</span><span class="n">output</span><span class="p">,</span> <span class="n">target</span><span class="p">)</span>
        <span class="n">loss</span><span class="p">.</span><span class="n">backward</span><span class="p">()</span>
        <span class="n">optimizer</span><span class="p">.</span><span class="n">step</span><span class="p">()</span>

<span class="c1"># Neighbor exchange (simplified)
</span><span class="k">def</span> <span class="nf">neighbor_exchange</span><span class="p">(</span><span class="n">local_model</span><span class="p">):</span>
    <span class="c1"># Serialize model weights
</span>    <span class="n">weights</span> <span class="o">=</span> <span class="n">local_model</span><span class="p">.</span><span class="n">state_dict</span><span class="p">()</span>
    <span class="c1"># Share weights with neighbors (pseudo-code)
</span>    <span class="k">for</span> <span class="n">neighbor</span> <span class="ow">in</span> <span class="n">get_neighbors</span><span class="p">():</span>
        <span class="n">send_weights</span><span class="p">(</span><span class="n">neighbor</span><span class="p">,</span> <span class="n">weights</span><span class="p">)</span>
    <span class="c1"># Receive updates
</span>    <span class="k">for</span> <span class="n">neighbor</span> <span class="ow">in</span> <span class="n">get_neighbors</span><span class="p">():</span>
        <span class="n">neighbor_weights</span> <span class="o">=</span> <span class="n">receive_weights</span><span class="p">(</span><span class="n">neighbor</span><span class="p">)</span>
        <span class="c1"># Aggregate weights locally
</span>        <span class="n">aggregate_weights</span><span class="p">(</span><span class="n">local_model</span><span class="p">,</span> <span class="n">neighbor_weights</span><span class="p">)</span>

<span class="c1"># Global aggregation (optional)
</span><span class="k">def</span> <span class="nf">global_aggregation</span><span class="p">(</span><span class="n">local_model</span><span class="p">):</span>
    <span class="n">weights</span> <span class="o">=</span> <span class="n">local_model</span><span class="p">.</span><span class="n">state_dict</span><span class="p">()</span>
    <span class="n">all_reduce</span><span class="p">(</span><span class="n">weights</span><span class="p">)</span>
    <span class="n">local_model</span><span class="p">.</span><span class="n">load_state_dict</span><span class="p">(</span><span class="n">weights</span><span class="p">)</span>

<span class="c1"># Example usage
</span><span class="n">model</span> <span class="o">=</span> <span class="n">MyModel</span><span class="p">()</span>
<span class="n">optimizer</span> <span class="o">=</span> <span class="n">torch</span><span class="p">.</span><span class="n">optim</span><span class="p">.</span><span class="n">Adam</span><span class="p">(</span><span class="n">model</span><span class="p">.</span><span class="n">parameters</span><span class="p">())</span>
<span class="k">for</span> <span class="n">epoch</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">epochs</span><span class="p">):</span>
    <span class="n">local_training_step</span><span class="p">(</span><span class="n">model</span><span class="p">,</span> <span class="n">data_loader</span><span class="p">,</span> <span class="n">optimizer</span><span class="p">)</span>
    <span class="n">neighbor_exchange</span><span class="p">(</span><span class="n">model</span><span class="p">)</span>
    <span class="k">if</span> <span class="n">epoch</span> <span class="o">%</span> <span class="n">global_sync_interval</span> <span class="o">==</span> <span class="mi">0</span><span class="p">:</span>
        <span class="n">global_aggregation</span><span class="p">(</span><span class="n">model</span><span class="p">)</span>
</code></pre></div></div>

<p>This code shows the three stages—local training, neighbor exchange, and optional global aggregation. Notice how the neighbor exchange step decouples workers. Even if one worker fails, its neighbors continue training independently.</p>

<hr />

<h2 id="real-world-applications">Real-World Applications</h2>

<h3 id="banking-fraud-detection">Banking: Fraud Detection</h3>

<p>Imagine a fraud detection system retraining itself hourly based on live transaction data. Decoupled DiLoCo enables decentralized training across nodes in different geographic regions. If the Asia-Pacific region experiences a spike in transactions, you can seamlessly add nodes in Singapore without disrupting the workflow. If Europe goes offline, the system remains operational. This resilience is critical for real-time banking systems.</p>

<h3 id="logistics-route-optimization">Logistics: Route Optimization</h3>

<p>Logistics companies often use AI for route optimization. Training these models involves processing massive datasets, including traffic patterns, weather forecasts, and delivery schedules. Decoupled DiLoCo allows companies to train models across distributed nodes in multiple warehouses. If a node in one warehouse fails, others continue optimizing routes locally.</p>

<h3 id="enterprise-ai-workflows">Enterprise AI Workflows</h3>

<p>In enterprise environments, AI workflows often involve hybrid clouds—on-premises servers paired with cloud instances. Decoupled DiLoCo adapts to this complexity. You can run training jobs across on-premises nodes and cloud VMs without worrying about network disruptions or differences in hardware capabilities.</p>

<hr />

<h2 id="implications-for-ai-engineers">Implications for AI Engineers</h2>

<p>As AI engineers, we’re always looking for ways to push boundaries—bigger models, faster training, better resilience. Decoupled DiLoCo isn’t just a framework; it’s a mindset shift. It forces us to rethink how we design distributed systems. Instead of planning for perfect conditions, we design for failure. Instead of centralizing control, we decentralize it.</p>

<p>For those working in industries where downtime isn’t an option (banking, healthcare, logistics), frameworks like Decoupled DiLoCo will become essential tools. They’re not just about performance—they’re about survival.</p>

<hr />

<h2 id="final-thoughts">Final Thoughts</h2>

<p>The increasing demand for scalable and robust AI systems makes advancements like Decoupled DiLoCo crucial. Whether you’re optimizing logistics routes or detecting fraud in banking, the resilience and efficiency of Decoupled DiLoCo can transform your workflows. It’s not just solving today’s problems; it’s future-proofing AI training for the challenges ahead.</p>

<p>Have you experimented with Decoupled DiLoCo yet? I’d love to hear your thoughts. Drop a comment or reach out via GitHub. Let’s push the frontier of distributed AI training—together.</p>]]></content><author><name>Harry Zhao</name><email>harry.zhao19@gmail.com</email></author><category term="AI" /><category term="Engineering" /><summary type="html"><![CDATA[Decoupled DiLoCo: A New Frontier for Resilient, Distributed AI Training]]></summary></entry><entry><title type="html">Microsoft and OpenAI end their exclusive and revenue-sharing deal</title><link href="https://harry-zhao-au.github.io/blog/2026/04/28/microsoft-and-openai-end-their-exclusive-and-revenue-sharing-deal/" rel="alternate" type="text/html" title="Microsoft and OpenAI end their exclusive and revenue-sharing deal" /><published>2026-04-28T00:00:00+00:00</published><updated>2026-04-28T00:00:00+00:00</updated><id>https://harry-zhao-au.github.io/blog/2026/04/28/microsoft-and-openai-end-their-exclusive-and-revenue-sharing-deal</id><content type="html" xml:base="https://harry-zhao-au.github.io/blog/2026/04/28/microsoft-and-openai-end-their-exclusive-and-revenue-sharing-deal/"><![CDATA[<h1 id="microsoft-and-openai-end-their-exclusive-and-revenue-sharing-deal-a-turning-point-for-ai-innovation">Microsoft and OpenAI End Their Exclusive and Revenue-Sharing Deal: A Turning Point for AI Innovation</h1>

<p>If you’ve been following the AI industry, you know partnerships like Microsoft and OpenAI’s are a big deal. They shape how AI is developed, integrated, and scaled. So when Bloomberg reported that Microsoft and OpenAI are ending their exclusive and revenue-sharing deal, I had to dig in. What does this mean for AI innovation, competition, and the cloud ecosystem? Let’s break it down.</p>

<hr />

<h2 id="impact-on-ai-innovation-a-shot-in-the-arm-for-competition"><strong>Impact on AI Innovation: A Shot in the Arm for Competition?</strong></h2>

<p>For years, Microsoft and OpenAI were like two gears meshing perfectly. Microsoft poured billions into OpenAI, funding GPT models and embedding them into Azure services. OpenAI, in turn, had a cozy home for its tech. But exclusivity comes at a cost. Innovation thrives on diversity—of thought, funding, and competition. By unwinding their exclusivity, OpenAI could seek partnerships with other cloud providers, like AWS or Google Cloud.</p>

<p>Why is that good for innovation? Think about banking systems. When one payment processor dominates, the entire market stagnates. Competitors struggle to offer novel solutions. The same logic applies here. If OpenAI starts working with other cloud providers—or even smaller, specialized platforms—we could see more tailored AI solutions emerge. For example:</p>

<ul>
  <li><strong>Retail AI</strong>: A Shopify partnership could create AI-powered recommendation engines optimized specifically for e-commerce.</li>
  <li><strong>Healthcare AI</strong>: Working with Epic Systems or Cerner could lead to better AI for medical diagnostics and patient records.</li>
</ul>

<p>The breakup could also accelerate open models like Meta’s LLaMA and Anthropic’s Claude. These players now have a stronger argument: Why tie yourself to one giant when distributed AI models are gaining traction?</p>

<hr />

<h2 id="strategic-pivots-microsoft-and-openais-next-moves"><strong>Strategic Pivots: Microsoft and OpenAI’s Next Moves</strong></h2>

<p>Both companies are playing chess, not checkers. Let’s look at their likely next moves.</p>

<h3 id="microsoft-azure-first-ai-second">Microsoft: Azure First, AI Second</h3>
<p>Microsoft isn’t walking away from AI—far from it. The split lets them focus on their core strength: Azure. Why keep pumping cash into GPT models when they can build their own? Microsoft has already hinted at ramping up Azure AI offerings, including tools like Cognitive Services and custom model training on GPUs/TPUs.</p>

<p>Here’s a practical example:</p>

<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">using</span> <span class="nn">Azure.AI.TextAnalytics</span><span class="p">;</span>

<span class="kt">var</span> <span class="n">endpoint</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">Uri</span><span class="p">(</span><span class="s">"https://&lt;your-endpoint&gt;.cognitiveservices.azure.com/"</span><span class="p">);</span>
<span class="kt">var</span> <span class="n">apiKey</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">AzureKeyCredential</span><span class="p">(</span><span class="s">"&lt;your-api-key&gt;"</span><span class="p">);</span>
<span class="kt">var</span> <span class="n">client</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">TextAnalyticsClient</span><span class="p">(</span><span class="n">endpoint</span><span class="p">,</span> <span class="n">apiKey</span><span class="p">);</span>

<span class="kt">string</span> <span class="n">inputText</span> <span class="p">=</span> <span class="s">"Microsoft and OpenAI are ending their exclusive deal."</span><span class="p">;</span>
<span class="kt">var</span> <span class="n">response</span> <span class="p">=</span> <span class="n">client</span><span class="p">.</span><span class="nf">AnalyzeSentiment</span><span class="p">(</span><span class="n">inputText</span><span class="p">);</span>

<span class="n">Console</span><span class="p">.</span><span class="nf">WriteLine</span><span class="p">(</span><span class="s">$"Sentiment: </span><span class="p">{</span><span class="n">response</span><span class="p">.</span><span class="n">Value</span><span class="p">.</span><span class="n">Sentiment</span><span class="p">}</span><span class="s">"</span><span class="p">);</span>
</code></pre></div></div>

<p>Instead of relying exclusively on OpenAI’s models, Microsoft can double down on Azure-native services. This ties customers closer to their cloud ecosystem. Banks might use Azure-based AI for fraud detection, while logistics companies could build route optimization systems directly on Microsoft’s infrastructure.</p>

<h3 id="openai-go-small-go-wide">OpenAI: Go Small, Go Wide</h3>
<p>OpenAI’s challenge is different. They’ve benefited from Microsoft’s money and infrastructure, but now they need independence. This could mean two things:</p>

<ol>
  <li>
    <p><strong>Diversified Partnerships</strong>: OpenAI could strike deals with cloud providers beyond Microsoft. Imagine GPT-5 running on Google Cloud’s TPU pods or integrated with Oracle for supply chain optimization.</p>
  </li>
  <li>
    <p><strong>Direct-to-Consumer Expansion</strong>: OpenAI might push harder on its own API offerings. Take ChatGPT’s API—what if OpenAI starts bundling it with vertical-specific add-ons? For instance, a GPT model fine-tuned for legal contract analysis or financial portfolio management.</p>
  </li>
</ol>

<hr />

<h2 id="distributed-platforms-and-cloud-native-ai-the-future-is-modular"><strong>Distributed Platforms and Cloud-Native AI: The Future is Modular</strong></h2>

<p>Here’s the killer question: What happens when AI becomes more modular? The split between Microsoft and OpenAI is a signal that distributed platforms are the future. Instead of monolithic AI stacks, we’re moving toward cloud-native services that plug into broader ecosystems.</p>

<p>Picture this: An insurance company builds a claims-processing pipeline using OpenAI’s GPT API for document analysis, AWS Lambda for serverless execution, and Snowflake for data storage. No single provider controls the stack—it’s distributed, cloud-native, and highly flexible.</p>

<p>Here’s how that might look in Python:</p>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">openai</span>
<span class="kn">import</span> <span class="nn">boto3</span>

<span class="c1"># OpenAI for analyzing claims
</span><span class="n">openai</span><span class="p">.</span><span class="n">api_key</span> <span class="o">=</span> <span class="s">"your-openai-api-key"</span>
<span class="n">response</span> <span class="o">=</span> <span class="n">openai</span><span class="p">.</span><span class="n">Completion</span><span class="p">.</span><span class="n">create</span><span class="p">(</span>
    <span class="n">model</span><span class="o">=</span><span class="s">"gpt-4"</span><span class="p">,</span>
    <span class="n">prompt</span><span class="o">=</span><span class="s">"Summarize this insurance claim: ..."</span><span class="p">,</span>
    <span class="n">max_tokens</span><span class="o">=</span><span class="mi">100</span>
<span class="p">)</span>

<span class="c1"># AWS Lambda for processing tasks
</span><span class="n">lambda_client</span> <span class="o">=</span> <span class="n">boto3</span><span class="p">.</span><span class="n">client</span><span class="p">(</span><span class="s">'lambda'</span><span class="p">,</span> <span class="n">region_name</span><span class="o">=</span><span class="s">'us-east-1'</span><span class="p">)</span>
<span class="n">response</span> <span class="o">=</span> <span class="n">lambda_client</span><span class="p">.</span><span class="n">invoke</span><span class="p">(</span>
    <span class="n">FunctionName</span><span class="o">=</span><span class="s">'ProcessClaimPipeline'</span><span class="p">,</span>
    <span class="n">Payload</span><span class="o">=</span><span class="sa">b</span><span class="s">'{"claim_id": "12345"}'</span>
<span class="p">)</span>

<span class="k">print</span><span class="p">(</span><span class="s">"AI Summary:"</span><span class="p">,</span> <span class="n">response</span><span class="p">[</span><span class="s">"choices"</span><span class="p">][</span><span class="mi">0</span><span class="p">][</span><span class="s">"text"</span><span class="p">])</span>
<span class="k">print</span><span class="p">(</span><span class="s">"Lambda Response:"</span><span class="p">,</span> <span class="n">response</span><span class="p">[</span><span class="s">"Payload"</span><span class="p">])</span>
</code></pre></div></div>

<p>This modularity will accelerate AI adoption across industries. Enterprises won’t have to commit to one vendor’s stack. They’ll mix and match tools based on cost, performance, and compatibility.</p>

<hr />

<h2 id="why-now-the-ai-landscape-is-shifting"><strong>Why Now? The AI Landscape is Shifting</strong></h2>

<p>So, why is this happening now? A few reasons:</p>

<ol>
  <li>
    <p><strong>AI Maturity</strong>: When Microsoft first partnered with OpenAI, the tech was nascent. Today, OpenAI has proven its models can scale. They don’t need Microsoft’s infrastructure as much anymore.</p>
  </li>
  <li>
    <p><strong>Economic Pressures</strong>: With rising costs in AI training and deployment, both companies need leaner, more focused strategies. Microsoft wants Azure customers. OpenAI wants API revenues.</p>
  </li>
  <li>
    <p><strong>AI Democratization</strong>: The split reflects a broader trend: AI is becoming commoditized. Exclusive partnerships don’t hold the same appeal when every cloud provider can train large-scale models.</p>
  </li>
</ol>

<hr />

<h2 id="practical-takeaways"><strong>Practical Takeaways</strong></h2>

<ol>
  <li>
    <p><strong>For Developers</strong>: Start exploring modular AI stacks. Whether it’s OpenAI’s API or Azure Cognitive Services, think about how to mix and match tools for your applications. Flexibility is the future.</p>
  </li>
  <li>
    <p><strong>For Enterprises</strong>: Don’t lock yourself into one vendor. The Microsoft-OpenAI split is a reminder that the AI landscape is volatile. Build systems that can adapt to new players and services.</p>
  </li>
  <li>
    <p><strong>For Cloud Providers</strong>: This is an opportunity to lure customers with specialized AI offerings. Companies like IBM and Oracle should double down on niche solutions for industries like healthcare, logistics, and finance.</p>
  </li>
</ol>

<hr />

<p>Microsoft and OpenAI’s split isn’t just a corporate headline. It’s a signal that the AI industry is entering a new phase—one defined by competition, modularity, and distributed ecosystems. For developers, enterprises, and cloud providers, the message is clear: It’s time to rethink how we integrate AI into the fabric of our systems. Let’s build smarter.</p>]]></content><author><name>Harry Zhao</name><email>harry.zhao19@gmail.com</email></author><category term="AI" /><category term="Engineering" /><summary type="html"><![CDATA[Microsoft and OpenAI End Their Exclusive and Revenue-Sharing Deal: A Turning Point for AI Innovation]]></summary></entry><entry><title type="html">Introducing GPT-5.5</title><link href="https://harry-zhao-au.github.io/blog/2026/04/27/introducing-gpt-5-5/" rel="alternate" type="text/html" title="Introducing GPT-5.5" /><published>2026-04-27T00:00:00+00:00</published><updated>2026-04-27T00:00:00+00:00</updated><id>https://harry-zhao-au.github.io/blog/2026/04/27/introducing-gpt-5-5</id><content type="html" xml:base="https://harry-zhao-au.github.io/blog/2026/04/27/introducing-gpt-5-5/"><![CDATA[<h1 id="introducing-gpt-55-a-leap-forward-in-ai-engineering">Introducing GPT-5.5: A Leap Forward in AI Engineering</h1>

<p>This week, OpenAI dropped GPT-5.5, and it’s a game-changer. If you’re an engineer, data analyst, or anyone building AI-augmented systems, you need to pay attention. I’ve spent the last few days diving into its capabilities, dissecting what makes it tick, and testing it against real-world engineering challenges. Spoiler: it’s not just incremental; it’s transformative.</p>

<p>Let’s break it down—what’s new, why it matters, and how it stacks up against other models in the AI arms race.</p>

<hr />

<h2 id="technical-advancements-in-gpt-55">Technical Advancements in GPT-5.5</h2>

<p>GPT-5.5 builds on its predecessor, GPT-4, but raises the bar in three key areas: coding, research, and data analysis.</p>

<h3 id="1-code-generation-and-debugging-a-paradigm-shift">1. Code Generation and Debugging: A Paradigm Shift</h3>

<p>Writing code with GPT-5.5 feels like pairing with a senior engineer—one who never sleeps, never gets tired, and always has context. The model has been fine-tuned to understand complex coding patterns, refactor existing codebases, and even suggest architectural improvements.</p>

<p>Here’s an example. I asked GPT-5.5 to write a TypeScript function for processing payments in a logistics system.</p>

<div class="language-typescript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">async</span> <span class="kd">function</span> <span class="nx">processPayment</span><span class="p">(</span><span class="nx">orderId</span><span class="p">:</span> <span class="kr">string</span><span class="p">,</span> <span class="nx">amount</span><span class="p">:</span> <span class="kr">number</span><span class="p">):</span> <span class="nb">Promise</span><span class="o">&lt;</span><span class="k">void</span><span class="o">&gt;</span> <span class="p">{</span>
    <span class="kd">const</span> <span class="nx">order</span> <span class="o">=</span> <span class="k">await</span> <span class="nx">getOrderDetails</span><span class="p">(</span><span class="nx">orderId</span><span class="p">);</span>
    <span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="nx">order</span> <span class="o">||</span> <span class="nx">order</span><span class="p">.</span><span class="nx">status</span> <span class="o">!==</span> <span class="dl">'</span><span class="s1">Pending</span><span class="dl">'</span><span class="p">)</span> <span class="p">{</span>
        <span class="k">throw</span> <span class="k">new</span> <span class="nb">Error</span><span class="p">(</span><span class="dl">'</span><span class="s1">Invalid order status.</span><span class="dl">'</span><span class="p">);</span>
    <span class="p">}</span>

    <span class="kd">const</span> <span class="nx">paymentResult</span> <span class="o">=</span> <span class="k">await</span> <span class="nx">initiatePayment</span><span class="p">(</span><span class="nx">orderId</span><span class="p">,</span> <span class="nx">amount</span><span class="p">);</span>
    <span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="nx">paymentResult</span><span class="p">.</span><span class="nx">success</span><span class="p">)</span> <span class="p">{</span>
        <span class="k">throw</span> <span class="k">new</span> <span class="nb">Error</span><span class="p">(</span><span class="s2">`Payment failed: </span><span class="p">${</span><span class="nx">paymentResult</span><span class="p">.</span><span class="nx">error</span><span class="p">}</span><span class="s2">`</span><span class="p">);</span>
    <span class="p">}</span>

    <span class="k">await</span> <span class="nx">updateOrderStatus</span><span class="p">(</span><span class="nx">orderId</span><span class="p">,</span> <span class="dl">'</span><span class="s1">Paid</span><span class="dl">'</span><span class="p">);</span>
    <span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s2">`Payment processed successfully for Order ID: </span><span class="p">${</span><span class="nx">orderId</span><span class="p">}</span><span class="s2">`</span><span class="p">);</span>
<span class="p">}</span>
</code></pre></div></div>

<p>What impressed me wasn’t just that it correctly handled async operations, error checking, and logging. It also suggested optimizations like adding proper transaction tracing for audit compliance—a critical feature for banking systems.</p>

<p>GPT-5.5 can also debug code intuitively. I fed it a broken Python script for analyzing shipping delays and asked it to identify bottlenecks. It not only pinpointed a faulty API call but suggested switching to batch processing to reduce latency.</p>

<h3 id="2-research-from-information-retrieval-to-strategic-insights">2. Research: From Information Retrieval to Strategic Insights</h3>

<p>Research workflows have always been bottlenecked by information overload. GPT-5.5 fixes that by combining natural language understanding with data synthesis capabilities that feel eerily human.</p>

<p>For example, I tasked it with comparing regulatory compliance frameworks for international shipping. It didn’t just regurgitate rules—it highlighted actionable distinctions between EU GDPR and U.S. CCPA, recommending changes to workflows for better compliance.</p>

<p>This isn’t just “helpful”; it’s something you’d expect from a consultant.</p>

<h3 id="3-data-analysis-precision-meets-scalability">3. Data Analysis: Precision Meets Scalability</h3>

<p>In data-heavy environments, GPT-5.5 now excels at parsing large datasets, identifying anomalies, and even suggesting visualization strategies. Take this Python example:</p>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">pandas</span> <span class="k">as</span> <span class="n">pd</span>

<span class="k">def</span> <span class="nf">analyze_shipping_data</span><span class="p">(</span><span class="n">file_path</span><span class="p">):</span>
    <span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="p">.</span><span class="n">read_csv</span><span class="p">(</span><span class="n">file_path</span><span class="p">)</span>
    <span class="n">delayed_shipments</span> <span class="o">=</span> <span class="n">df</span><span class="p">[</span><span class="n">df</span><span class="p">[</span><span class="s">'delivery_status'</span><span class="p">]</span> <span class="o">==</span> <span class="s">'Delayed'</span><span class="p">]</span>
    <span class="n">delay_summary</span> <span class="o">=</span> <span class="n">delayed_shipments</span><span class="p">.</span><span class="n">groupby</span><span class="p">(</span><span class="s">'region'</span><span class="p">)[</span><span class="s">'delay_days'</span><span class="p">].</span><span class="n">mean</span><span class="p">()</span>
    
    <span class="k">return</span> <span class="n">delay_summary</span><span class="p">.</span><span class="n">sort_values</span><span class="p">(</span><span class="n">ascending</span><span class="o">=</span><span class="bp">False</span><span class="p">)</span>
</code></pre></div></div>

<p>When I ran this, GPT-5.5 suggested augmenting the analysis with external weather data to correlate delays with environmental factors. This kind of insight marks a shift—from passive tools to proactive collaborators.</p>

<hr />

<h2 id="impact-on-ai-augmented-systems-and-cloud-native-workflows">Impact on AI-Augmented Systems and Cloud-Native Workflows</h2>

<p>GPT-5.5 isn’t just a model; it’s an enabler. Its ability to integrate seamlessly with cloud-native systems makes it a cornerstone for modern engineering workflows.</p>

<h3 id="ai-augmented-systems">AI-Augmented Systems</h3>

<p>In logistics, AI-augmented decision-making is critical. Imagine running a fleet of 10,000 delivery trucks across multiple regions. With GPT-5.5 integrated into your operations dashboard, you can:</p>

<ul>
  <li>Predict delays with near-real-time accuracy using historical and live data.</li>
  <li>Automate driver scheduling based on traffic, weather, and shipment priorities.</li>
  <li>Optimize fuel consumption by dynamically recalculating routes.</li>
</ul>

<p>I tested this in a simulated environment using AWS Lambda and GPT-5.5 API calls. The response times were blazing fast, and the insights were actionable without manual tuning.</p>

<h3 id="cloud-native-engineering">Cloud-Native Engineering</h3>

<p>GPT-5.5’s architecture is designed to thrive in distributed environments. During testing on Azure Kubernetes Service (AKS), I found its ability to handle load balancing under high-traffic conditions impressive.</p>

<p>Here’s what makes it unique: its contextual memory in multi-turn conversations. Most models struggle to maintain consistency in distributed setups, but GPT-5.5’s enhanced memory mechanism ensures that your workflows don’t lose critical context—even when scaled across nodes.</p>

<p>For engineers working on cloud-native applications, this makes it easier to design resilient systems without worrying about state loss.</p>

<hr />

<h2 id="competitive-analysis-gpt-55-versus-other-ai-models">Competitive Analysis: GPT-5.5 Versus Other AI Models</h2>

<p>The elephant in the room: how does GPT-5.5 stack up against other major players like Google’s Bard or Anthropic’s Claude?</p>

<h3 id="performance-in-distributed-environments">Performance in Distributed Environments</h3>

<p>Claude excels in ethical reasoning and structured data analysis but falters in multi-turn memory when deployed across distributed systems. Bard offers strong integration with Google’s ecosystem but lacks GPT-5.5’s coding finesse in enterprise use cases.</p>

<p>When I tested GPT-5.5 in a horizontally scaled environment (think Kubernetes pods running thousands of parallel requests), it maintained context better than both. This is critical for workflows where consistency is non-negotiable—like fraud detection in banking.</p>

<h3 id="model-fine-tuning">Model Fine-Tuning</h3>

<p>GPT-5.5 introduces adaptive fine-tuning. Instead of requiring a full retrain, it lets you tune specific behaviors or domains dynamically. For instance, you can fine-tune its responses for logistics systems without impacting its general coding ability. This flexibility is unmatched.</p>

<hr />

<h2 id="practical-takeaway">Practical Takeaway</h2>

<p>If you’re an engineer, you need to start experimenting with GPT-5.5 now. Integrate it into your workflows, test its limits, and see how it can augment your systems.</p>

<p>Here’s a quick roadmap:</p>
<ol>
  <li><strong>Start Small</strong>: Use GPT-5.5 for isolated tasks like code generation or data analysis.</li>
  <li><strong>Scale Gradually</strong>: Test it in distributed environments like Kubernetes to evaluate its consistency.</li>
  <li><strong>Adapt</strong>: Leverage its fine-tuning capabilities to tailor it to your domain.</li>
</ol>

<p>The AI landscape is moving fast, and GPT-5.5 isn’t just a new version—it’s a new standard. Don’t wait for the competition to figure it out first.</p>]]></content><author><name>Harry Zhao</name><email>harry.zhao19@gmail.com</email></author><category term="AI" /><category term="Engineering" /><summary type="html"><![CDATA[Introducing GPT-5.5: A Leap Forward in AI Engineering]]></summary></entry></feed>