<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>AI on DecipherMiddleware</title><link>https://blog.deciphermiddleware.in/tags/ai/</link><description>Recent content in AI on DecipherMiddleware</description><generator>Hugo -- gohugo.io</generator><language>en-us</language><lastBuildDate>Wed, 10 Jun 2026 00:00:00 +0530</lastBuildDate><atom:link href="https://blog.deciphermiddleware.in/tags/ai/index.xml" rel="self" type="application/rss+xml"/><item><title>CLI Coding Agents Meet MuleSoft MCP Server</title><link>https://blog.deciphermiddleware.in/posts/cli-coding-agents-meet-mule-mcp/</link><pubDate>Wed, 10 Jun 2026 00:00:00 +0530</pubDate><guid>https://blog.deciphermiddleware.in/posts/cli-coding-agents-meet-mule-mcp/</guid><description>&lt;p&gt;As an Integration and DevOps engineer, I&amp;rsquo;ve spent countless hours staring at black and white screens — &lt;code&gt;vim&lt;/code&gt;, &lt;code&gt;vi&lt;/code&gt;, &lt;code&gt;nano&lt;/code&gt;, writing, running, testing, and shipping scripts, all from &lt;strong&gt;one home&lt;/strong&gt;:&lt;/p&gt;
&lt;h3 id="the-og"&gt;The OG &lt;a href="#the-og" class="anchor"&gt;&lt;i class="fa-solid fa-anchor"&gt;&lt;/i&gt;&lt;/a&gt;&lt;/h3&gt;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-sh" data-lang="sh"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s2"&gt;&amp;#34;terminal&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Coding agents just moved in.&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-sh" data-lang="sh"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;## Antigravity&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;agy
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;## GitHub Copilot&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;copilot
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;After experimenting with &lt;strong&gt;IDE&lt;/strong&gt; coding agents — building MuleSoft applications using &lt;strong&gt;MuleSoft Vibes/ACB&lt;/strong&gt; right inside VS Code — I wanted to take it further. Back to where I&amp;rsquo;m most comfortable.&lt;/p&gt;
&lt;p&gt;Let&amp;rsquo;s spin up a CLI coding agent, and arm it with MuleSoft DX MCP Server to interact with Anypoint Platform - straight from the terminal.&lt;/p&gt;
&lt;h2 id="table-of-contents"&gt;Table of Contents &lt;a href="#table-of-contents" class="anchor"&gt;&lt;i class="fa-solid fa-anchor"&gt;&lt;/i&gt;&lt;/a&gt;&lt;/h2&gt;&lt;ul&gt;
&lt;li&gt;&lt;a href="#mulesoft-dx-mcp-server"&gt;MuleSoft DX MCP Server&lt;/a&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href="#features"&gt;Features&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;&lt;a href="#prerequisites"&gt;Prerequisites&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#connected-app-creation"&gt;Connected App Creation&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#the-og-terminal"&gt;The OG: Terminal&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#cli-coding-agents"&gt;CLI Coding Agents&lt;/a&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href="#antigravity"&gt;Antigravity&lt;/a&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href="#mcp-server-configuration"&gt;MCP Server configuration&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#moment-of-truth"&gt;Moment of truth&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;&lt;a href="#copilot"&gt;Copilot&lt;/a&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href="#mcp-server-configuration-1"&gt;MCP Server configuration&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#moment-of-truth-1"&gt;Moment of truth&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;hr&gt;
&lt;h2 id="mulesoft-dx-mcp-server"&gt;MuleSoft DX MCP Server &lt;a href="#mulesoft-dx-mcp-server" class="anchor"&gt;&lt;i class="fa-solid fa-anchor"&gt;&lt;/i&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;MuleSoft DX MCP Server supports API design, application deployment, and asset management&lt;/p&gt;
&lt;h3 id="features"&gt;Features &lt;a href="#features" class="anchor"&gt;&lt;i class="fa-solid fa-anchor"&gt;&lt;/i&gt;&lt;/a&gt;&lt;/h3&gt;&lt;ul&gt;
&lt;li&gt;API specification development&lt;/li&gt;
&lt;li&gt;Application creation and configuration&lt;/li&gt;
&lt;li&gt;Agent network creation&lt;/li&gt;
&lt;li&gt;Asset publication in Exchange&lt;/li&gt;
&lt;li&gt;Instance deployment and management to CloudHub 2.0 and Runtime Fabric&lt;/li&gt;
&lt;li&gt;Application security&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;These features look very promising for a CLI agent. Let&amp;rsquo;s set them up.&lt;/p&gt;
&lt;p&gt;&lt;img src="https://blog.deciphermiddleware.in/images/blogger/cb1bd4f698.gif" alt=""&gt;&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id="prerequisites"&gt;Prerequisites &lt;a href="#prerequisites" class="anchor"&gt;&lt;i class="fa-solid fa-anchor"&gt;&lt;/i&gt;&lt;/a&gt;&lt;/h2&gt;&lt;ul&gt;
&lt;li&gt;Enable Agentforce for Anypoint platform - &lt;a href="https://docs.mulesoft.com/access-management/enabling-agentforce"&gt;Ref&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Install Node.js 20 or later - &lt;a href="https://nodejs.org/en/download"&gt;Ref&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Organization admin access to create a connected app - &lt;a href="https://docs.mulesoft.com/access-management/connected-apps-overview"&gt;Ref&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;hr&gt;
&lt;h2 id="connected-app-creation"&gt;Connected App Creation &lt;a href="#connected-app-creation" class="anchor"&gt;&lt;i class="fa-solid fa-anchor"&gt;&lt;/i&gt;&lt;/a&gt;&lt;/h2&gt;&lt;ul&gt;
&lt;li&gt;Log in to Anypoint Platform.&lt;/li&gt;
&lt;li&gt;Under Access Management, click &lt;strong&gt;Connected Apps&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;Click &lt;strong&gt;Create App&lt;/strong&gt;. Add a name for the connected app.&lt;/li&gt;
&lt;li&gt;Select &lt;strong&gt;App acts on its own behalf (client credentials)&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;Click on Add Scopes. Add the scopes mentioned in the below table.&lt;/li&gt;
&lt;/ul&gt;
&lt;table&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;th&gt;Anypoint Code Builder&lt;/th&gt;
&lt;td&gt;Mule Developer Generative AI User&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;th&gt;Anypoint Monitoring&lt;/th&gt;
&lt;td&gt;Monitoring Viewer&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;th&gt;Anypoint Code Builder&lt;/th&gt;
&lt;td&gt;Mule Developer Generative AI User&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;th rowspan=3&gt;API Manager&lt;/th&gt;
&lt;td&gt;Manage APIs Configuration&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Manage APIs Configuration&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;View APIs Configuration&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;th rowspan=4&gt;Exchange&lt;/th&gt;
&lt;td&gt;Exchange Administrator&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Exchange Contributor&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Exchange Creator&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Exchange Viewer&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;th rowspan=2&gt;General&lt;/th&gt;
&lt;td&gt;View Connected Applications&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;View Organization&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;th&gt;OpenID&lt;/th&gt;
&lt;td&gt;Profile&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;th rowspan=4&gt;Runtime Manager&lt;/th&gt;
&lt;td&gt;Cloudhub Network Viewer&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Create Applications&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Read Applications&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Read Runtime Fabrics&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;ul&gt;
&lt;li&gt;Click on &lt;strong&gt;Save&lt;/strong&gt; to save the new connected app.&lt;/li&gt;
&lt;/ul&gt;
&lt;hr&gt;
&lt;h2 id="the-og-terminal"&gt;The OG: Terminal &lt;a href="#the-og-terminal" class="anchor"&gt;&lt;i class="fa-solid fa-anchor"&gt;&lt;/i&gt;&lt;/a&gt;&lt;/h2&gt;&lt;ul&gt;
&lt;li&gt;Now open &lt;code&gt;The OG: Terminal&lt;/code&gt;. Set the environment variables for the terminal session.&lt;/li&gt;
&lt;/ul&gt;
&lt;h4 id="macoslinux"&gt;macOS/Linux &lt;a href="#macoslinux" class="anchor"&gt;&lt;i class="fa-solid fa-anchor"&gt;&lt;/i&gt;&lt;/a&gt;&lt;/h4&gt;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-sh" data-lang="sh"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nb"&gt;export&lt;/span&gt; &lt;span class="nv"&gt;ANYPOINT_CLIENT_ID&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;&amp;lt;CLIENT_ID&amp;gt;&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nb"&gt;export&lt;/span&gt; &lt;span class="nv"&gt;ANYPOINT_CLIENT_SECRET&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;&amp;lt;CLIENT_SECRET&amp;gt;&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h5 id="optional"&gt;Optional &lt;a href="#optional" class="anchor"&gt;&lt;i class="fa-solid fa-anchor"&gt;&lt;/i&gt;&lt;/a&gt;&lt;/h5&gt;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-sh" data-lang="sh"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nb"&gt;export&lt;/span&gt; &lt;span class="nv"&gt;ANYPOINT_REGION&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;&amp;lt;REGION_NAME&amp;gt;&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h4 id="windows"&gt;Windows &lt;a href="#windows" class="anchor"&gt;&lt;i class="fa-solid fa-anchor"&gt;&lt;/i&gt;&lt;/a&gt;&lt;/h4&gt;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-pwsh" data-lang="pwsh"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nv"&gt;$env:ANYPOINT_CLIENT_ID&lt;/span&gt;&lt;span class="p"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;&amp;lt;CLIENT_ID&amp;gt;&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nv"&gt;$env:ANYPOINT_CLIENT_SECRET&lt;/span&gt;&lt;span class="p"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;&amp;lt;CLIENT_SECRET&amp;gt;&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h5 id="optional-1"&gt;Optional &lt;a href="#optional-1" class="anchor"&gt;&lt;i class="fa-solid fa-anchor"&gt;&lt;/i&gt;&lt;/a&gt;&lt;/h5&gt;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-sh" data-lang="sh"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nv"&gt;$env&lt;/span&gt;:ANYPOINT_REGION&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;&amp;lt;REGION_NAME&amp;gt;&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;hr&gt;
&lt;h2 id="cli-coding-agents"&gt;CLI Coding Agents &lt;a href="#cli-coding-agents" class="anchor"&gt;&lt;i class="fa-solid fa-anchor"&gt;&lt;/i&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;Setup is complete on the &lt;strong&gt;Anypoint&lt;/strong&gt; side and the &lt;strong&gt;env variables&lt;/strong&gt; are set in the terminal. Let&amp;rsquo;s make the terminal smarter and equip it with MuleSoft DX MCP Server.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Antigravity&lt;/li&gt;
&lt;li&gt;Copilot&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;img src="https://blog.deciphermiddleware.in/images/wp/giphy-4.webp" alt=""&gt;&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id="antigravity"&gt;Antigravity &lt;a href="#antigravity" class="anchor"&gt;&lt;i class="fa-solid fa-anchor"&gt;&lt;/i&gt;&lt;/a&gt;&lt;/h3&gt;&lt;ul&gt;
&lt;li&gt;Install Antigravity CLI following the &lt;a href="https://antigravity.google/download#antigravity-cli"&gt;link&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Run the below command to start CLI. Log in using your google/gemini account.&lt;/li&gt;
&lt;/ul&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-sh" data-lang="sh"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;agy
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;img src="https://blog.deciphermiddleware.in/images/screenshots/gemini-1.png" alt=""&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Type &lt;code&gt;/mcp&lt;/code&gt;, press Enter.&lt;/li&gt;
&lt;/ul&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-sh" data-lang="sh"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;MCP Servers
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; No MCP servers configured.
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;Keyboard: ↑/↓ Navigate enter Actions
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;esc to cancel
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;ul&gt;
&lt;li&gt;As of now, no MCP servers are configured for antigravity-cli.&lt;/li&gt;
&lt;/ul&gt;
&lt;h4 id="mcp-server-configuration"&gt;MCP Server configuration &lt;a href="#mcp-server-configuration" class="anchor"&gt;&lt;i class="fa-solid fa-anchor"&gt;&lt;/i&gt;&lt;/a&gt;&lt;/h4&gt;&lt;ul&gt;
&lt;li&gt;Antigravity lets you setup MCP server as Global setup or Workspace Local setup.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;strong&gt;Global:&lt;/strong&gt; &lt;code&gt;~/.gemini/antigravity-cli/mcp_config.json&lt;/code&gt;&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Workspace local setups:&lt;/strong&gt; &lt;code&gt;.agents/mcp_config.json&lt;/code&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Update &lt;code&gt;mcp_config.json&lt;/code&gt; as below.&lt;/li&gt;
&lt;/ul&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-json" data-lang="json"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="p"&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="nt"&gt;&amp;#34;mcpServers&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="nt"&gt;&amp;#34;mulesoft&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="nt"&gt;&amp;#34;command&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;&amp;#34;npx&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="nt"&gt;&amp;#34;args&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;-y&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;&amp;#34;mulesoft-mcp-server&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;&amp;#34;start&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="p"&gt;}&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="p"&gt;}&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;ul&gt;
&lt;li&gt;Make sure that &lt;a href="#the-og-terminal"&gt;env variables&lt;/a&gt; were properly set for the terminal session.&lt;/li&gt;
&lt;li&gt;Let&amp;rsquo;s start &lt;strong&gt;antigravity&lt;/strong&gt;.&lt;/li&gt;
&lt;/ul&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-sh" data-lang="sh"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;agy
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;ul&gt;
&lt;li&gt;Type &lt;code&gt;/mcp&lt;/code&gt;, press Enter.&lt;/li&gt;
&lt;/ul&gt;
&lt;pre tabindex="0"&gt;&lt;code&gt;MCP Servers
Plugins (~/.gemini/antigravity-cli/plugins)
&amp;gt; ✓ mulesoft Tools: create_mule_project, generate_mule_flow, create_mcp_server, deploy_mule_application, list_applications, +16 more
Keyboard: ↑/↓ Navigate enter Actions
&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;If you see the message similar to above then you are all set and ready to perform operations &amp;#x1f604;&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;But wait, is it too easy????&lt;/p&gt;&lt;/blockquote&gt;
&lt;h4 id="lets-get-the-hands-dirty-"&gt;Let&amp;rsquo;s get the hands dirty 😈 &lt;a href="#lets-get-the-hands-dirty-" class="anchor"&gt;&lt;i class="fa-solid fa-anchor"&gt;&lt;/i&gt;&lt;/a&gt;&lt;/h4&gt;&lt;blockquote&gt;
&lt;p&gt;My initial prompts didn&amp;rsquo;t work and, it was not able to list the MCP tools &amp;#x1f61e; Countless hours and tokens wasted 😫!!!&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;I kept on trying&amp;hellip;. I kept trying, and then I changed the &lt;code&gt;/model&lt;/code&gt; to &lt;code&gt;Claude Sonnet 4.6&lt;/code&gt;, and prompted:&lt;/p&gt;
&lt;pre tabindex="0"&gt;&lt;code&gt;list mcp tools
&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;A good output &amp;#x1f60e;&lt;/p&gt;
&lt;p&gt;&lt;a href="https://blog.deciphermiddleware.in/images/screenshots/gemini-2.png"&gt;&lt;img src="https://blog.deciphermiddleware.in/images/screenshots/gemini-2.png" alt=""&gt;&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;Another prompt&lt;/p&gt;
&lt;pre tabindex="0"&gt;&lt;code&gt;List deployed applications in dev environment.
&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;Another success &amp;#x1f60e;&lt;/p&gt;
&lt;p&gt;&lt;a href="https://blog.deciphermiddleware.in/images/screenshots/gemini-3.png"&gt;&lt;img src="https://blog.deciphermiddleware.in/images/screenshots/gemini-3.png" alt=""&gt;&lt;/a&gt;&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id="copilot"&gt;Copilot &lt;a href="#copilot" class="anchor"&gt;&lt;i class="fa-solid fa-anchor"&gt;&lt;/i&gt;&lt;/a&gt;&lt;/h3&gt;&lt;ul&gt;
&lt;li&gt;Install the copilot following the &lt;a href="https://github.com/features/copilot/cli"&gt;link&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Run the below command to check if copilot is working.&lt;/li&gt;
&lt;/ul&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-sh" data-lang="sh"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;copilot
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;ul&gt;
&lt;li&gt;Type &lt;code&gt;/login&lt;/code&gt; and press enter. Follow instructions to log in to your GitHub account and activate GitHub Copilot.&lt;/li&gt;
&lt;li&gt;To view list of available MCP servers type &lt;code&gt;/mcp show&lt;/code&gt; and press enter.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;a href="https://blog.deciphermiddleware.in/images/screenshots/copilot-1.png"&gt;&lt;img src="https://blog.deciphermiddleware.in/images/screenshots/copilot-1.png" alt=""&gt;&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;As of now only GitHub MCP server is enabled.&lt;/p&gt;
&lt;h4 id="mcp-server-configuration-1"&gt;MCP Server configuration &lt;a href="#mcp-server-configuration-1" class="anchor"&gt;&lt;i class="fa-solid fa-anchor"&gt;&lt;/i&gt;&lt;/a&gt;&lt;/h4&gt;&lt;ul&gt;
&lt;li&gt;Type &lt;code&gt;/mcp add&lt;/code&gt; and press enter. Fill in details as per below table.&lt;/li&gt;
&lt;/ul&gt;
&lt;table&gt;
&lt;tr&gt;
&lt;th&gt;Server Name:&lt;/th&gt;
&lt;td&gt;mulesoft-dx&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;th&gt;Server Type:&lt;/th&gt;
&lt;td&gt;STDIO&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;th&gt;Command to start the server:&lt;/th&gt;
&lt;td&gt;npx -y mulesoft-mcp-server start&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;th&gt;Environment variables:&lt;/th&gt;
&lt;td&gt;Add the env variables of ANYPOINT_CLIENT_ID and ANYPOINT_CLIENT_SECRET if not configured in terminal env else keep it blank.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;th&gt;Tools:&lt;/th&gt;
&lt;td&gt;*&lt;/td&gt;
&lt;/tr&gt;
&lt;/table&gt;
&lt;p&gt;&lt;a href="https://blog.deciphermiddleware.in/images/screenshots/copilot-2.png"&gt;&lt;img src="https://blog.deciphermiddleware.in/images/screenshots/copilot-2.png" alt=""&gt;&lt;/a&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;ctrl+s&lt;/code&gt; to save the MCP server configuration.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;a href="https://blog.deciphermiddleware.in/images/screenshots/copilot-3.png"&gt;&lt;img src="https://blog.deciphermiddleware.in/images/screenshots/copilot-3.png" alt=""&gt;&lt;/a&gt;&lt;/p&gt;
&lt;h4 id="moment-of-truth"&gt;Moment of truth &lt;a href="#moment-of-truth" class="anchor"&gt;&lt;i class="fa-solid fa-anchor"&gt;&lt;/i&gt;&lt;/a&gt;&lt;/h4&gt;&lt;h5 id="prompt"&gt;Prompt &lt;a href="#prompt" class="anchor"&gt;&lt;i class="fa-solid fa-anchor"&gt;&lt;/i&gt;&lt;/a&gt;&lt;/h5&gt;&lt;pre tabindex="0"&gt;&lt;code&gt;List deployed applications in dev environment.
&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;&lt;a href="https://blog.deciphermiddleware.in/images/screenshots/copilot-4.png"&gt;&lt;img src="https://blog.deciphermiddleware.in/images/screenshots/copilot-4.png" alt=""&gt;&lt;/a&gt;&lt;/p&gt;
&lt;hr&gt;
&lt;blockquote&gt;
&lt;p&gt;Now, CLI agent has got MuleSoft DX MCP Server.&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;&lt;a href="https://blog.deciphermiddleware.in/images/blogger/33b97e90f7.gif"&gt;&lt;img src="https://blog.deciphermiddleware.in/images/blogger/33b97e90f7.gif" alt=""&gt;&lt;/a&gt;&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id="references"&gt;References &lt;a href="#references" class="anchor"&gt;&lt;i class="fa-solid fa-anchor"&gt;&lt;/i&gt;&lt;/a&gt;&lt;/h2&gt;&lt;ul&gt;
&lt;li&gt;&lt;a href="https://docs.mulesoft.com/access-management/enabling-agentforce"&gt;https://docs.mulesoft.com/access-management/enabling-agentforce&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://docs.mulesoft.com/access-management/connected-apps-overview"&gt;https://docs.mulesoft.com/access-management/connected-apps-overview&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://docs.mulesoft.com/mulesoft-mcp-server/getting-started"&gt;https://docs.mulesoft.com/mulesoft-mcp-server/getting-started&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://antigravity.google/download#antigravity-cli"&gt;https://antigravity.google/download#antigravity-cli&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/features/copilot/cli"&gt;https://github.com/features/copilot/cli&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://antigravity.google/docs/cli-plugins#model-context-protocol-mcp"&gt;https://antigravity.google/docs/cli-plugins#model-context-protocol-mcp&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/features/copilot/cli"&gt;https://github.com/features/copilot/cli&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;</description></item><item><title>Installing AI models Localhost</title><link>https://blog.deciphermiddleware.in/posts/installing-ai-models-locally/</link><pubDate>Mon, 16 Feb 2026 00:00:00 +0000</pubDate><guid>https://blog.deciphermiddleware.in/posts/installing-ai-models-locally/</guid><description>&lt;p&gt;&lt;em&gt;&lt;center&gt;Everything works on my Localhost&lt;/center&gt;&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;  One day, I started wondering, is it possible to run AI models on a normal system where I play games? I recalled that I have a NVDIA graphics card in my desktop. I started digging into what I can do. I got so many doubts. &lt;strong&gt;Will the desktop be able to support? If yes, where do I start? Is it going to crash the system?&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;&lt;a href="https://blog.deciphermiddleware.in/images/img/ai-models-banner.jpg"&gt;&lt;img src="https://blog.deciphermiddleware.in/images/img/ai-models-banner.jpg" alt=""&gt;&lt;/a&gt;&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;Generated By AI (ChatGPT)&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;So many questions. Let&amp;rsquo;s ask gen AI (ChatGPT/Gemini&amp;hellip;etc).&lt;/p&gt;
&lt;p&gt;On putting the query. It started giving me answers that need huge servers and configurations. No! No!&amp;hellip;. !!! But do I have that much computing power? I need the answers that fit my existing computing resources.&lt;/p&gt;
&lt;p&gt;First things first, I grabbed a piece of paper. Oh wait! I know this works, but in today&amp;rsquo;s digital world. VsCode becomes the new notepad :P. I gathered a couple of pieces of information.&lt;/p&gt;
&lt;h2 id="desktop-configurations"&gt;Desktop Configurations &lt;a href="#desktop-configurations" class="anchor"&gt;&lt;i class="fa-solid fa-anchor"&gt;&lt;/i&gt;&lt;/a&gt;&lt;/h2&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Component&lt;/th&gt;
&lt;th&gt;Configurations&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;CPU&lt;/td&gt;
&lt;td&gt;i5 11400 11th Generation&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;RAM&lt;/td&gt;
&lt;td&gt;32 GB&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;SSD&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;GPU&lt;/td&gt;
&lt;td&gt;MSI GeForce GTX 1650 VENTUS XS OC Nvidia Graphic Card&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;OS&lt;/td&gt;
&lt;td&gt;Windows 11&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;blockquote&gt;
&lt;p&gt;Configurations, as per me looks decent. Will it be able to support any LMs? Answer is &lt;em&gt;&lt;strong&gt;YES&lt;/strong&gt;&lt;/em&gt;. Let&amp;rsquo;s give it a &lt;strong&gt;try&lt;/strong&gt;.&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;&lt;img src="https://blog.deciphermiddleware.in/images/blogger/c528afff99.gif" alt=""&gt;&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id="pre-requisites"&gt;Pre-requisites &lt;a href="#pre-requisites" class="anchor"&gt;&lt;i class="fa-solid fa-anchor"&gt;&lt;/i&gt;&lt;/a&gt;&lt;/h2&gt;&lt;h3 id="1-update-windows"&gt;1. Update Windows &lt;a href="#1-update-windows" class="anchor"&gt;&lt;i class="fa-solid fa-anchor"&gt;&lt;/i&gt;&lt;/a&gt;&lt;/h3&gt;&lt;ul&gt;
&lt;li&gt;Update operating system. I am using Windows, and performed a Windows update.&lt;/li&gt;
&lt;/ul&gt;
&lt;h3 id="2-update-gpu-drivers"&gt;2. Update GPU Drivers &lt;a href="#2-update-gpu-drivers" class="anchor"&gt;&lt;i class="fa-solid fa-anchor"&gt;&lt;/i&gt;&lt;/a&gt;&lt;/h3&gt;&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;Update GPU drivers. I have an NVidia GPU, so updating the &lt;a href="https://www.nvidia.com/en-in/drivers/nvidia-update/"&gt;NVidia GPU drivers&lt;/a&gt;.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Enable the CUDA cores. Download the CUDA toolkit.
&lt;br/&gt;&lt;a href="https://developer.nvidia.com/cuda/toolkit"&gt;https://developer.nvidia.com/cuda/toolkit&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;If the GPU is older, like mine, use the link below to find the compute compatibility version for the CUDA toolkit.
&lt;a href="https://developer.nvidia.com/cuda/gpus"&gt;https://developer.nvidia.com/cuda/gpus&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;For the &lt;strong&gt;GTX 1650&lt;/strong&gt;, it supports version &lt;strong&gt;7.5&lt;/strong&gt;.&lt;br/&gt; &lt;a href="https://developer.nvidia.com/cuda-75-downloads-archive"&gt;https://developer.nvidia.com/cuda-75-downloads-archive&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Run the command in cmd or PowerShell below to check if the drivers and the CUDA toolkit are installed.&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-PowerShell" data-lang="PowerShell"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nb"&gt;nvidia-smi&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;strong&gt;Output will look like this.&lt;/strong&gt;&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-powershell" data-lang="powershell"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="p"&gt;+-----------------------------------------------------------------------------------------+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="p"&gt;|&lt;/span&gt; &lt;span class="nb"&gt;NVIDIA-SMI&lt;/span&gt; &lt;span class="mf"&gt;591.86&lt;/span&gt; &lt;span class="n"&gt;Driver&lt;/span&gt; &lt;span class="n"&gt;Version&lt;/span&gt;&lt;span class="err"&gt;:&lt;/span&gt; &lt;span class="mf"&gt;591.86&lt;/span&gt; &lt;span class="n"&gt;CUDA&lt;/span&gt; &lt;span class="n"&gt;Version&lt;/span&gt;&lt;span class="err"&gt;:&lt;/span&gt; &lt;span class="mf"&gt;13.1&lt;/span&gt; &lt;span class="p"&gt;|&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="p"&gt;+-----------------------------------------+------------------------+----------------------+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="p"&gt;|&lt;/span&gt; &lt;span class="n"&gt;GPU&lt;/span&gt; &lt;span class="n"&gt;Name&lt;/span&gt; &lt;span class="nb"&gt;Driver-Model&lt;/span&gt; &lt;span class="p"&gt;|&lt;/span&gt; &lt;span class="nb"&gt;Bus-Id&lt;/span&gt; &lt;span class="n"&gt;Disp&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="py"&gt;A&lt;/span&gt; &lt;span class="p"&gt;|&lt;/span&gt; &lt;span class="n"&gt;Volatile&lt;/span&gt; &lt;span class="n"&gt;Uncorr&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt; &lt;span class="n"&gt;ECC&lt;/span&gt; &lt;span class="p"&gt;|&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="p"&gt;|&lt;/span&gt; &lt;span class="n"&gt;Fan&lt;/span&gt; &lt;span class="n"&gt;Temp&lt;/span&gt; &lt;span class="n"&gt;Perf&lt;/span&gt; &lt;span class="n"&gt;Pwr&lt;/span&gt;&lt;span class="err"&gt;:&lt;/span&gt;&lt;span class="n"&gt;Usage&lt;/span&gt;&lt;span class="p"&gt;/&lt;/span&gt;&lt;span class="n"&gt;Cap&lt;/span&gt; &lt;span class="p"&gt;|&lt;/span&gt; &lt;span class="nb"&gt;Memory-Usage&lt;/span&gt; &lt;span class="p"&gt;|&lt;/span&gt; &lt;span class="nb"&gt;GPU-Util&lt;/span&gt; &lt;span class="n"&gt;Compute&lt;/span&gt; &lt;span class="n"&gt;M&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt; &lt;span class="p"&gt;|&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="p"&gt;|&lt;/span&gt; &lt;span class="p"&gt;|&lt;/span&gt; &lt;span class="p"&gt;|&lt;/span&gt; &lt;span class="n"&gt;MIG&lt;/span&gt; &lt;span class="n"&gt;M&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt; &lt;span class="p"&gt;|&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="p"&gt;|=========================================+========================+======================|&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="p"&gt;|&lt;/span&gt; &lt;span class="mf"&gt;0&lt;/span&gt; &lt;span class="n"&gt;NVIDIA&lt;/span&gt; &lt;span class="n"&gt;GeForce&lt;/span&gt; &lt;span class="n"&gt;GTX&lt;/span&gt; &lt;span class="mf"&gt;1650&lt;/span&gt; &lt;span class="n"&gt;WDDM&lt;/span&gt; &lt;span class="p"&gt;|&lt;/span&gt; &lt;span class="mf"&gt;00000000&lt;/span&gt;&lt;span class="err"&gt;:&lt;/span&gt;&lt;span class="mf"&gt;01&lt;/span&gt;&lt;span class="err"&gt;:&lt;/span&gt;&lt;span class="mf"&gt;00.0&lt;/span&gt; &lt;span class="n"&gt;On&lt;/span&gt; &lt;span class="p"&gt;|&lt;/span&gt; &lt;span class="n"&gt;N&lt;/span&gt;&lt;span class="p"&gt;/&lt;/span&gt;&lt;span class="n"&gt;A&lt;/span&gt; &lt;span class="p"&gt;|&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="p"&gt;|&lt;/span&gt; &lt;span class="mf"&gt;40&lt;/span&gt;&lt;span class="p"&gt;%&lt;/span&gt; &lt;span class="n"&gt;36C&lt;/span&gt; &lt;span class="n"&gt;P8&lt;/span&gt; &lt;span class="n"&gt;15W&lt;/span&gt; &lt;span class="p"&gt;/&lt;/span&gt; &lt;span class="n"&gt;90W&lt;/span&gt; &lt;span class="p"&gt;|&lt;/span&gt; &lt;span class="n"&gt;681MiB&lt;/span&gt; &lt;span class="p"&gt;/&lt;/span&gt; &lt;span class="n"&gt;4096MiB&lt;/span&gt; &lt;span class="p"&gt;|&lt;/span&gt; &lt;span class="mf"&gt;4&lt;/span&gt;&lt;span class="p"&gt;%&lt;/span&gt; &lt;span class="k"&gt;Default&lt;/span&gt; &lt;span class="p"&gt;|&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="p"&gt;|&lt;/span&gt; &lt;span class="p"&gt;|&lt;/span&gt; &lt;span class="p"&gt;|&lt;/span&gt; &lt;span class="n"&gt;N&lt;/span&gt;&lt;span class="p"&gt;/&lt;/span&gt;&lt;span class="n"&gt;A&lt;/span&gt; &lt;span class="p"&gt;|&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="p"&gt;+-----------------------------------------+------------------------+----------------------+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="p"&gt;+-----------------------------------------------------------------------------------------+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="p"&gt;|&lt;/span&gt; &lt;span class="n"&gt;Processes&lt;/span&gt;&lt;span class="err"&gt;:&lt;/span&gt; &lt;span class="p"&gt;|&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="p"&gt;|&lt;/span&gt; &lt;span class="n"&gt;GPU&lt;/span&gt; &lt;span class="nb"&gt;GI &lt;/span&gt; &lt;span class="n"&gt;CI&lt;/span&gt; &lt;span class="n"&gt;PID&lt;/span&gt; &lt;span class="nb"&gt;Type &lt;/span&gt; &lt;span class="k"&gt;Process&lt;/span&gt; &lt;span class="n"&gt;name&lt;/span&gt; &lt;span class="n"&gt;GPU&lt;/span&gt; &lt;span class="n"&gt;Memory&lt;/span&gt; &lt;span class="p"&gt;|&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="p"&gt;|&lt;/span&gt; &lt;span class="n"&gt;ID&lt;/span&gt; &lt;span class="n"&gt;ID&lt;/span&gt; &lt;span class="n"&gt;Usage&lt;/span&gt; &lt;span class="p"&gt;|&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="p"&gt;|=========================================================================================|&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="p"&gt;|&lt;/span&gt; &lt;span class="mf"&gt;0&lt;/span&gt; &lt;span class="n"&gt;N&lt;/span&gt;&lt;span class="p"&gt;/&lt;/span&gt;&lt;span class="n"&gt;A&lt;/span&gt; &lt;span class="n"&gt;N&lt;/span&gt;&lt;span class="p"&gt;/&lt;/span&gt;&lt;span class="n"&gt;A&lt;/span&gt; &lt;span class="mf"&gt;1816&lt;/span&gt; &lt;span class="n"&gt;C&lt;/span&gt;&lt;span class="p"&gt;+&lt;/span&gt;&lt;span class="n"&gt;G&lt;/span&gt; &lt;span class="p"&gt;...&lt;/span&gt;&lt;span class="n"&gt;8bbwe&lt;/span&gt;&lt;span class="p"&gt;\&lt;/span&gt;&lt;span class="n"&gt;PhoneExperienceHost&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="py"&gt;exe&lt;/span&gt; &lt;span class="n"&gt;N&lt;/span&gt;&lt;span class="p"&gt;/&lt;/span&gt;&lt;span class="n"&gt;A&lt;/span&gt; &lt;span class="p"&gt;|&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="p"&gt;|&lt;/span&gt; &lt;span class="p"&gt;|&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="p"&gt;+-----------------------------------------------------------------------------------------+&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h3 id="3-wsl-installation"&gt;3. WSL Installation &lt;a href="#3-wsl-installation" class="anchor"&gt;&lt;i class="fa-solid fa-anchor"&gt;&lt;/i&gt;&lt;/a&gt;&lt;/h3&gt;&lt;p&gt;Since I am running Windows, I need a Linux distribution to run an LM locally. To do so, I have options such as Docker, a virtual machine or WSL. To take advantage of GPUs in a virtual environment, WSL is the best option.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Check current WSL distros installed on the system.&lt;/li&gt;
&lt;/ol&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-powershell" data-lang="powershell"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;wsl&lt;/span&gt; &lt;span class="p"&gt;-&lt;/span&gt;&lt;span class="n"&gt;-list&lt;/span&gt; &lt;span class="p"&gt;-&lt;/span&gt;&lt;span class="n"&gt;-verbose&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;ol start="2"&gt;
&lt;li&gt;Install &lt;strong&gt;Ubuntu 22.04&lt;/strong&gt; using &lt;strong&gt;WSL 2&lt;/strong&gt;.&lt;/li&gt;
&lt;/ol&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-powershell" data-lang="powershell"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;wsl&lt;/span&gt; &lt;span class="p"&gt;-&lt;/span&gt;&lt;span class="n"&gt;-install&lt;/span&gt; &lt;span class="n"&gt;-d&lt;/span&gt; &lt;span class="n"&gt;Ubuntu&lt;/span&gt;&lt;span class="p"&gt;-&lt;/span&gt;&lt;span class="mf"&gt;22.04&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;ol start="3"&gt;
&lt;li&gt;After downloading the distros, it will ask for username and password. Provide &lt;strong&gt;username&lt;/strong&gt; and &lt;strong&gt;password&lt;/strong&gt;,which will be used to login.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;strong&gt;Check installation&lt;/strong&gt;&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-powershell" data-lang="powershell"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;wsl&lt;/span&gt; &lt;span class="p"&gt;-&lt;/span&gt;&lt;span class="n"&gt;-list&lt;/span&gt; &lt;span class="p"&gt;-&lt;/span&gt;&lt;span class="n"&gt;-verbose&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="n"&gt;NAME&lt;/span&gt; &lt;span class="n"&gt;STATE&lt;/span&gt; &lt;span class="n"&gt;VERSION&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="p"&gt;*&lt;/span&gt; &lt;span class="n"&gt;Ubuntu&lt;/span&gt;&lt;span class="p"&gt;-&lt;/span&gt;&lt;span class="mf"&gt;22.04&lt;/span&gt; &lt;span class="n"&gt;Stopped&lt;/span&gt; &lt;span class="mf"&gt;2&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;hr&gt;
&lt;details&gt;
&lt;p&gt;&lt;strong&gt;&lt;summary&gt;Moving installation to other directory&lt;/summary&gt;&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Export the installation to tar, to move the installation to another directory.&lt;/li&gt;
&lt;/ul&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-powershell" data-lang="powershell"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;wsl&lt;/span&gt; &lt;span class="p"&gt;-&lt;/span&gt;&lt;span class="n"&gt;-export&lt;/span&gt; &lt;span class="n"&gt;Ubuntu&lt;/span&gt;&lt;span class="p"&gt;-&lt;/span&gt;&lt;span class="mf"&gt;22.04&lt;/span&gt; &lt;span class="n"&gt;D:&lt;/span&gt;&lt;span class="p"&gt;\&lt;/span&gt;&lt;span class="n"&gt;AI&lt;/span&gt;&lt;span class="p"&gt;\&lt;/span&gt;&lt;span class="n"&gt;ubuntu&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="py"&gt;tar&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;ul&gt;
&lt;li&gt;Unregister the current installation from the list of installed distros.&lt;/li&gt;
&lt;/ul&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-powershell" data-lang="powershell"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;wsl&lt;/span&gt; &lt;span class="p"&gt;-&lt;/span&gt;&lt;span class="n"&gt;-unregister&lt;/span&gt; &lt;span class="n"&gt;Ubuntu&lt;/span&gt;&lt;span class="p"&gt;-&lt;/span&gt;&lt;span class="mf"&gt;22.04&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;ul&gt;
&lt;li&gt;Import to another location.&lt;/li&gt;
&lt;/ul&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-powershell" data-lang="powershell"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;wsl&lt;/span&gt; &lt;span class="p"&gt;-&lt;/span&gt;&lt;span class="n"&gt;-import&lt;/span&gt; &lt;span class="n"&gt;Ubuntu&lt;/span&gt;&lt;span class="p"&gt;-&lt;/span&gt;&lt;span class="mf"&gt;22.04&lt;/span&gt; &lt;span class="n"&gt;D:&lt;/span&gt;&lt;span class="p"&gt;\&lt;/span&gt;&lt;span class="n"&gt;AI&lt;/span&gt;&lt;span class="p"&gt;\&lt;/span&gt;&lt;span class="n"&gt;ubuntu&lt;/span&gt; &lt;span class="n"&gt;D:&lt;/span&gt;&lt;span class="p"&gt;\&lt;/span&gt;&lt;span class="n"&gt;AI&lt;/span&gt;&lt;span class="p"&gt;\&lt;/span&gt;&lt;span class="n"&gt;ubuntu&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="py"&gt;tar&lt;/span&gt; &lt;span class="p"&gt;-&lt;/span&gt;&lt;span class="n"&gt;-version&lt;/span&gt; &lt;span class="mf"&gt;2&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/details&gt;
&lt;hr&gt;
&lt;ol start="4"&gt;
&lt;li&gt;Login to Ubuntu using PowerShell.&lt;/li&gt;
&lt;/ol&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-powershell" data-lang="powershell"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;wsl&lt;/span&gt; &lt;span class="n"&gt;-d&lt;/span&gt; &lt;span class="n"&gt;Ubuntu&lt;/span&gt;&lt;span class="p"&gt;-&lt;/span&gt;&lt;span class="mf"&gt;22.04&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;ol start="5"&gt;
&lt;li&gt;Update the default user.&lt;/li&gt;
&lt;/ol&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-sh" data-lang="sh"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nb"&gt;echo&lt;/span&gt; -e &lt;span class="s2"&gt;&amp;#34;[user]\ndefault=your_username&amp;#34;&lt;/span&gt; &lt;span class="p"&gt;|&lt;/span&gt; sudo tee /etc/wsl.conf
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;ol start="6"&gt;
&lt;li&gt;Restart WSL.&lt;/li&gt;
&lt;/ol&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-powershell" data-lang="powershell"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;wsl&lt;/span&gt; &lt;span class="p"&gt;-&lt;/span&gt;&lt;span class="n"&gt;-shutdown&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;ol start="7"&gt;
&lt;li&gt;Login to Ubuntu using PowerShell.&lt;/li&gt;
&lt;/ol&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-powershell" data-lang="powershell"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;wsl&lt;/span&gt; &lt;span class="n"&gt;-d&lt;/span&gt; &lt;span class="n"&gt;Ubuntu&lt;/span&gt;&lt;span class="p"&gt;-&lt;/span&gt;&lt;span class="mf"&gt;22.04&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;ol start="8"&gt;
&lt;li&gt;Check if drivers are installed correctly inside WSL/Ubuntu. Run the below command inside the same window.&lt;/li&gt;
&lt;/ol&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-sh" data-lang="sh"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;nvidia-smi
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;hr&gt;
&lt;p&gt;Okay,! Now the desktop is ready to install and run &lt;strong&gt;LMs&lt;/strong&gt;. But which &lt;strong&gt;one&lt;/strong&gt; to run? How to &lt;strong&gt;interact&lt;/strong&gt; with LM? How to manage LMs locally?&lt;/p&gt;
&lt;p&gt;&lt;img src="https://blog.deciphermiddleware.in/images/blogger/86807ebb0c.gif" alt=""&gt;&lt;/p&gt;
&lt;h2 id="install-ollama"&gt;Install Ollama &lt;a href="#install-ollama" class="anchor"&gt;&lt;i class="fa-solid fa-anchor"&gt;&lt;/i&gt;&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;Let&amp;rsquo;s use &lt;a href="https://ollama.com/"&gt;Ollama&lt;/a&gt; 😎. Docs: &lt;a href="https://docs.ollama.com/"&gt;https://docs.ollama.com/&lt;/a&gt;&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Run below command to install Ollama.&lt;/li&gt;
&lt;/ol&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-sh" data-lang="sh"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;curl -fsSL https://ollama.com/install.sh &lt;span class="p"&gt;|&lt;/span&gt; sh
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;ol start="2"&gt;
&lt;li&gt;Verify installation.&lt;/li&gt;
&lt;/ol&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-sh" data-lang="sh"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;ollama --version
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;ol start="3"&gt;
&lt;li&gt;Create a directory to save ollama models. Create an environment variable to point to the new location.&lt;/li&gt;
&lt;/ol&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-sh" data-lang="sh"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;mkdir &amp;lt;path-to-store-ollama-models&amp;gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;nano ~/.bashrc
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nb"&gt;export&lt;/span&gt; &lt;span class="nv"&gt;OLLAMA_MODELS&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&amp;lt;path-to-store-ollama-models&amp;gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nb"&gt;source&lt;/span&gt; ~/.bashrc
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;## Validate if environment variable is set properly&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="nv"&gt;$OLLAMA_MODELS&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;ol start="4"&gt;
&lt;li&gt;Start Ollama&lt;/li&gt;
&lt;/ol&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-sh" data-lang="sh"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;ollama serve
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;ol start="5"&gt;
&lt;li&gt;Now Ollama will start running. In another terminal, verify that Ollama is running:&lt;/li&gt;
&lt;/ol&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-sh" data-lang="sh"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;ollama -v
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;blockquote&gt;
&lt;p&gt;ollama version is 0.15.2&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;&lt;img src="https://blog.deciphermiddleware.in/images/blogger/33b97e90f7.gif" alt=""&gt;&lt;/p&gt;
&lt;p&gt;I am done with the hard part 😎😎😎.&lt;/p&gt;
&lt;ol start="6"&gt;
&lt;li&gt;Now, let&amp;rsquo;s download the AI model. To be on the safe side, I started small by downloading a vector embedding model &lt;strong&gt;(nomic-embed-text)&lt;/strong&gt;.&lt;/li&gt;
&lt;/ol&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-sh" data-lang="sh"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;ollama pull nomic-embed-text
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;ol start="7"&gt;
&lt;li&gt;Test the downloaded model.&lt;/li&gt;
&lt;/ol&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-sh" data-lang="sh"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;curl --location &lt;span class="s1"&gt;&amp;#39;http://localhost:11434/api/embeddings&amp;#39;&lt;/span&gt; &lt;span class="se"&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="se"&gt;&lt;/span&gt;--header &lt;span class="s1"&gt;&amp;#39;Content-Type: application/json&amp;#39;&lt;/span&gt; &lt;span class="se"&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="se"&gt;&lt;/span&gt;--data &lt;span class="s1"&gt;&amp;#39;{
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="s1"&gt; &amp;#34;model&amp;#34;: &amp;#34;nomic-embed-text&amp;#34;,
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="s1"&gt; &amp;#34;prompt&amp;#34;: &amp;#34;deciphermiddleware&amp;#34;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="s1"&gt; }&amp;#39;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;blockquote&gt;
&lt;p&gt;A vector output is generated. A successful test!!!&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;How much LM work I can offload to the GPU depends on VRAM a lot. Since the GTX 1650 has only 4GB VRAM, it will not allow large models to run on the GPU. Thus, models will run on a shared basis between CPU and GPU. Let me try a 3b parameter model &lt;a href="https://ollama.com/library/llama3.2"&gt;llama3.2&lt;/a&gt;.&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-sh" data-lang="sh"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;ollama pull llama3.2
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;ollama run llama3.2
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;strong&gt;Output&lt;/strong&gt;&lt;/p&gt;
&lt;pre tabindex="0"&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; hi
How can I assist you today?
&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;&lt;strong&gt;DEBUG INFO&lt;/strong&gt;&lt;/p&gt;
&lt;pre tabindex="0"&gt;&lt;code&gt;load_tensors: loading model tensors, this can take a while... (mmap = true)
load_tensors: offloading 26 repeating layers to GPU
load_tensors: offloaded 26/29 layers to GPU
load_tensors: CPU_Mapped model buffer size = 1918.35 MiB
load_tensors: CUDA0 model buffer size = 1488.14 MiB
llama_context: constructing llama_context
llama_context: n_seq_max = 1
llama_context: n_ctx = 4096
llama_context: n_ctx_seq = 4096
llama_context: n_batch = 512
llama_context: n_ubatch = 512
llama_context: causal_attn = 1
llama_context: flash_attn = auto
llama_context: kv_unified = false
llama_context: freq_base = 500000.0
llama_context: freq_scale = 1
llama_context: n_ctx_seq (4096) &amp;lt; n_ctx_train (131072) -- the full capacity of the model will not be utilized
llama_context: CPU output buffer size = 0.50 MiB
llama_kv_cache: CPU KV buffer size = 32.00 MiB
llama_kv_cache: CUDA0 KV buffer size = 416.00 MiB
llama_kv_cache: size = 448.00 MiB ( 4096 cells, 28 layers, 1/1 seqs), K (f16): 224.00 MiB, V (f16): 224.00 MiB
llama_context: Flash Attention was auto, set to enabled
llama_context: CUDA0 compute buffer size = 588.73 MiB
llama_context: CUDA_Host compute buffer size = 14.01 MiB
llama_context: graph nodes = 875
llama_context: graph splits = 29 (with bs=512), 3 (with bs=1)
&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;&lt;img src="https://media4.giphy.com/media/v1.Y2lkPTc5MGI3NjExb2wxZm56ZHZxbDE2bGdwNm44eXd1Ymhlbnp1NjFpaXVqdjJnYjl6YyZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw/111ebonMs90YLu/giphy.gif" alt=""&gt;&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;Now, let&amp;rsquo;s start using the &lt;strong&gt;AI models&lt;/strong&gt; and explore them more. But that will be for some other day.&lt;/p&gt;
&lt;p&gt;I hope you like the journey. Please share your valuable feedback. &amp;#x1f60a;&amp;#x1f60a;&amp;#x1f60a;&lt;/p&gt;
&lt;p&gt;&lt;img src="https://media4.giphy.com/media/v1.Y2lkPTc5MGI3NjExcWI2ejE1OG10ajcxeDFpb2s2MzZxZjluaThhZm9ncjRzdDBoaHJsdCZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw/9ssDWl7DA5XGbIY8OB/giphy.gif" alt=""&gt;&lt;/p&gt;</description></item></channel></rss>