ChatGPT-like LLMs to run locally
Artificial intelligence

5 ChatGPT-Like LLMs to Run Locally on Your PC

The best ChatGPT-like LLMs to run locally on your PC


Quick Findings

  • Gemma (by Google AI) is lightweight and the best option for lower-spec-ed PCs.
  • With a minimum of 16GB of RAM, you may opt for a pre-trained model like Llama 3.

1. Gemma – Lightweight, Open Model

Gemma, developed by Google DeepMind, is a great choice for running a ChatGPT-like LLM locally on your PC because of its open-source nature, lightweight architecture, good performance, and user-friendly tools.

Important Features

Model Size2B or 7B Parameters
Open SourceYes
Frameworks SupportedJAX, PyTorch, TensorFlow
Deployment OptionsLocal, Cloud
  • Lightweight Architecture: In contrast to certain other LLMs with enormous parameter counts, Gemma comes in two sizes, 2 billion and 7 billion parameters, making it possible for it to operate on a larger variety of personal computers due to its lower resource requirements as compared to models with tens or hundreds of billions of parameters.
  • Open Source and Free to Use: Gemma is an open-source project, meaning anyone can use and alter the tools and code. Compared to closed-source solutions, this transparency makes installation and modification simpler.
  • High-quality Performance: Gemma performs competitively on various tasks often associated with ChatGPT-like models, such as question answering, translation, and text production, even though it is lightweight. This guarantees a positive user experience without compromising functionality.
  • Multiple Framework Supported: Using native Keras 3.0, Gemma provides toolchains for inference (using the model) across common frameworks such as JAX, PyTorch, and TensorFlow. Because of this versatility, you can run the model in any setting that suits you.
  • Simple Deployment: Gemma offers pre-trained and instruction-tuned variations to customize your options. It also provides simple deployment options using Google Kubernetes Engine (GKE) and Vertex AI on various platforms, including your workstation, laptop, and Google Cloud.

2. Jurassic-1 – Large, Sophisticated Language Model

Although Jurassic-1 is a strong LLM for local usage, casual users may find it less suited because of its high resource requirements and technical intricacy.

With its remarkable capabilities, privacy features, and offline functionality, Jurassic-1 is an excellent choice for anyone with the necessary hardware and technological know-how. It comes in two sizes (Jumbo and Large), with Jumbo being the most sophisticated version.

Important Features

Model Size (Parameters)178B (Jumbo), 7B (Large)
PerformanceHigh (Jumbo), Competitive (Large)
Hardware RequirementsHigh (Jumbo), Moderate (Large)
Technical DifficultyHigh
  • Outstanding Performance: There are two versions of Jurassic-1 available: J1-Jumbo (178B parameters) and J1-Large (7B parameters). With performance on many jobs matching that of considerably larger models, J1-Jumbo is a great option for heavy users.
  • Available and Open-Source: Like Gemma, Jurassic-1 is available and open-source, enabling access and possible codebase modification. This transparency is quite useful for academics and developers. The model comes pre-trained on a large dataset, prepared to take on tasks such as question answering, translation, and text generation.
  • Customization Potential: Using your data to train this model, you can fine-tune Jurassic-1 to make it more suitable for particular activities or domains. This can be especially helpful for research projects or specialized applications.

3. Hugging Face Transformers (Mistral-7B-Instruct-v0.3) -Customizable & Efficient Models
Hugging face

For those looking for a locally runnable LLM that focuses on following instructions, Hugging Face Transformers (Mistral-7B-Instruct-v0.3) is a great choice. Users with technological knowledge and a compatible PC can benefit from its integration with Hugging Face and moderate size.

Important Features

Model Size (Parameters)7 Billion
PerformanceGood for following instructions
Pre-trained & Instruction-TunedYes (Instruction-tuned)
User-friendly ToolsIntegrates with Hugging Face Transformers
Technical DifficultyModerate
  • Fine-tuned to Follow Instructions: Mistral-7B-Instruct-v0.3 is specially optimized for obeying instructions and accomplishing tasks as instructed. This makes it ideal for situations where you want the LLM to follow your instructions and carry out particular tasks, such as crafting various unique text styles or responding to inquiries in a specific way.
  • Relatively Smaller Size: Mistral-7B-Instruct-v0.3 is within a reasonable size range for local use at 7 billion parameters. It may be a viable choice for PCs with strong GPUs because it takes less processing power than giants like Jurassic-1 (178B).
  • Hugging Face Integration: Mistral-7B-Instruct-v0.3’s smooth integration with the Hugging Face Transformers library is a major benefit. This library makes setting up and interacting with the model locally easier by offering comprehensive documentation and user-friendly tools. This can be especially helpful for people familiar with the Hugging Face environment.

4. GPT-J 6B – Open-Source Model

The GPT-J 6B strikes a great balance between local use and performance. Because of its open-source design, well-established community, and intuitive features, it is a good alternative for PC users looking for a flexible and easy-to-use LLM experience.

Important Features

Model Size (Parameters)6 Billion
PerformanceCompetitive for size
Hardware RequirementsModerate GPU recommended
Technical DifficultyModerate
Community & ToolsGrowing community, user-friendly tools available
  • Manageable Size and Open-Source: GPT-J is an open-source model that enables code exploration and possible modification. Most importantly, it lies in a sweet location with a 6 billion parameter size. It is massive enough to function well on various tasks, such as text production and question answering, but not as massive as Jurassic-1 (178 billion). As a result, it requires fewer system resources, which makes it a more practical choice for PCs with decent but not particularly exceptional specs.
  • Competitive Performance: GPT-J performs competitively despite its moderate size. Studies indicate that it produces outcomes on par with much larger models on some criteria. This implies that you should anticipate high-quality results for assignments such as information summarization, code development, and creative writing.
  • Well-Established Community and Tools: GPT-J enjoys the advantages of an expanding developer and enthusiast community. This translates to resources and technologies that are easily accessible for local model execution. Frameworks like Transformers and libraries like text-generation-webui provide user-friendly interfaces and streamlined workflows for configuring and working with GPT-J.
  • Active Development: GPT-J’s project is continuously improved, and problem patches are made. This guarantees that your local LLM experience is current and uses recent developments.

5. Llama 3 – Pre-trained or Instruction-Tuned Model

Llama 3 is another great chatGPT-like LLM with a distinctive value proposition for local use. Because of its efficiency and low resource requirements, it is an excellent choice for people with less powerful computers or those who value a lightweight and user-friendly experience.

Important Features

Model Size (Parameters)300 Million – 1.3 Billion
PerformanceDecent for size
Hardware RequirementsLow (CPU or low-power GPU)
Technical DifficultyModerate
Development FocusEfficiency
  • Lightweight Design: With between 300 million and 1.3 billion characteristics, Llama 3 is remarkably small compared to its larger competitors. Compared to models like Jurassic-1, this small footprint has far lower computing demands. Because of this, Llama 3 is a great option for desktop computers with less powerful GPUs or CPUs.
  • Sound Performance: Llama 3 exhibits remarkable ability for its weight class despite its small size. According to research, it performs admirably on various tasks, such as text production, question answering, and code completion. It offers a decent compromise between convenience and capability for casual users or those with limited hardware resources.
  • Focus on Efficiency: One of Llama 3’s primary design goals is to operate as effectively as possible on low-power devices. This means less powerful devices may operate more smoothly and consume less power. This efficiency-focused approach fits in nicely with the constraints of local LLM execution.
  • Open-Source and Active Development: The codebase of Llama 3 is open-source, enabling exploration and possible modification. It also gains from Google AI’s continuous development, which guarantees regular enhancements and bug corrections. This guarantees that your local LLM experience is current.

Leave a Response

David Ogbor
David is a tech guru with extensive knowledge in technical articles. He is passionate about writing and presents technical articles in an easy-to-understand format for easy comprehension. He aims to present easy solutions for day-to-day problems encountered while using PC. In his spare time, he likes traveling, playing sports, and singing.