Home/ Questions/Q 18336
In Process
Jhero23
  • 0
Asked: 2024-05-13T15:24:21+00:00

How can I run chatgpt using LLMs locally ?

  • 0

We’re now at a point where we can run (some thing like ChatGPT) using LLMs locally on our machines, taking advantage of the GPU usage for systems such as Apple Silicon, NVIDIA graphics, and all the other brands of GPU out there. Guess what? you don’t have to go through the entire setup of these models to make it work.

From my personal experience, while doing it the unoptimised way, I found out that for example Llama3 which is about 140GB takes about 5 hours to respond to one single prompt. Yes, that’s obvious if it is using CPU. However, there is a better and faster way to do it, with a few tools I will introduce you to in this writeup.

First, you want to get openwebui from the github page, clone this project to your machine

git clone https://github.com/open-webui/open-webui.git

This is an open source project that will give you the interface and management platform that looks like chatgpt from openAI at the moment. You can do many things with it, such as register an account locally, this data is stored on your machine (don’t be scared, your data is not going to anyone), sign in, save prompt, download more models, combine models and use them in your prompts.

To be able to use this, we need another tool called Ollama. Head on to the official web page and download it for your preferred operating system type. Once you do this, ollama should be running on your machine, the way to test this is to go to this localhost on your browser.

http://localhost:11434

You should get a message saying “Ollama is running“. Now you’re ready to pull in the models. On the model list page, you will find a list of the models that have been optimized for GPU usage in ollama. To get one, use the command

ollama pull llama3, ollama pull llama2, ollama pull phi3

These three commands are for pulling these three models, replace the model name you want and pull it. Once that is done, now you can run it using as you can see in the image below

ollama run llama3, ollama run llama2, ollama run phi3

Open openwebui project you cloned in your favourite IDE, in this case, I’m using Visual Studio Code. In the terminal/ command prompt of your VScode, navigate to the backend folder of the project using the command

cd backend

Now, you need to create a new conda environment, this will keep your python packages seperated just for this purpoose only. Assumption here is that you have conda installed on your machine, if you don’t, head on to the official page and select your OS type. Once you have conda installed, create a new environment using the following command

conda create --name openwebui python=3.11

// openwebui could be any name depending on you and then once its is created, activate it by using the command conda activate openwebui. You are now ready to install your python packages, run the following command:

pip install -r requirements.txt -U

The assumption here is that you have pip installed on your machine, and have used it before (if you don’t have pip, download it from here). You can see the requirements.txt file inside the backend folder, it contains all the packages needed. This command installs all the packages required for the project to function properly, -U means upgrade any package that is outdated.

You’re almost ready, you are done with the backend of the application, the front end part of the app (openwebui) is written in js so you need to install all the node dependencies. Do so by running

npm install
npm run build

Before running these commands, make sure that you have node installed, I recommend node version 18 and above, you can find it here, select your version and download, if you did it successfully, you can type node --version on your machine and it should respond with the appropriate version you installed.
The frontend setup is done.

To start the application, go to the backend folder in the terminal/Command Prompt and enter this command
bash start.sh

You should see something like the image below, go to your browser and enter the url http://0.0.0.0:8080. Then you register to the application, select a model to start using (hopefully you should see the list of models you pulled earlier)

Good Luck!!!!, Now you have LLM locally on your machine

Share

    2 Answers

    1. Exploring the Real Estate Market in Nainital: A Guide to Buying Property in the Scenic Hill Station
      Nestled in the Kumaon region of Uttarakhand, Nainital is one of India’s most picturesque hill stations, known for its serene lakes, lush greenery, and breathtaking views of the Himalayas. Its popularity as a tourist destination is well-established, but in recent years, Nainital has also emerged as a coveted location for real estate investment. Whether you are considering buying a holiday home, investing in a rental property, or looking to settle in the hills, Nainital offers a unique blend of natural beauty and real estate opportunities. This blog will explore various aspects of buying property in Nainital, including the types of properties available, the benefits of investing, legal considerations, and tips for prospective buyers.
      Why Invest in Nainital?
      Nainital’s charm lies in its stunning landscape, temperate climate, and rich cultural heritage. These factors make it an attractive destination not just for tourists but also for those looking to invest in property. Here are some reasons why Nainital is an appealing choice for real estate investment:

      1. Scenic Beauty and Lifestyle: Nainital is renowned for its natural beauty, including the famous Naini Lake, which serves as the town’s centerpiece. The hill station offers a peaceful and serene environment, ideal for those looking to escape the hustle and bustle of city life. Owning a property here means waking up to panoramic views of the mountains, enjoying pleasant weather year-round, and living amidst nature.
      2. Tourism and Rental Income: As a popular tourist destination, Nainital sees a steady influx of visitors throughout the year. This creates an opportunity for property owners to generate rental income by leasing out their homes to tourists. Short-term vacation rentals are particularly lucrative during peak seasons, offering attractive returns on investment.
      3. Proximity to Major Cities: Nainital’s accessibility from major cities like Delhi, Lucknow, and Dehradun adds to its appeal as a real estate hotspot. The hill station is well-connected by road, making it a convenient getaway for weekend travelers and those seeking a second home away from the city.
      4. Long-term Appreciation: The demand for properties in hill stations like Nainital has been on the rise, leading to consistent appreciation in property values. Investing in real estate here can provide long-term capital gains, especially as more people seek refuge in quieter, more natural settings.

      Types of Properties Available in Nainital
      Nainital offers a diverse range of properties to suit different needs and budgets. Whether you’re looking for a cozy cottage, a modern villa, or a piece of land to build your dream home, Nainital has something for everyone. Here are some of the most common types of properties available:

      1. Residential Villas and Cottages: Many buyers are drawn to Nainital for its charming cottages and villas, often built in traditional architectural styles with stone and wood elements. These properties are ideal for those looking for a second home or a vacation retreat. Modern villas with contemporary amenities are also available, catering to those who prefer a blend of tradition and modernity.
      2. Apartments: For those seeking a more compact living space, apartments in Nainital offer a practical option. Many new developments feature modern apartments with scenic views, making them popular among both buyers and renters.
      3. Plots of Land: If you prefer to build your own home, purchasing a plot of land is a great option. Nainital offers various plots in different sizes and locations, allowing buyers to design and construct their homes according to their preferences.
      4. Commercial Properties: With tourism being a major industry in Nainital, commercial properties such as hotels, guesthouses, and shops are in demand. Investing in a commercial property can be a profitable venture, particularly in prime locations close to tourist attractions.

      Legal Considerations for Buying Property in Nainital
      Before purchasing property in Nainital, it is crucial to be aware of the legal regulations and procedures involved. Here are some key legal considerations:

      1. Land Ownership Rules: In Uttarakhand, non-residents are allowed to buy land, but there are restrictions on the size of the plot. The ceiling for agricultural land is 250 square meters for non-agricultural purposes. It is essential to verify the type of land and ensure that the transaction complies with state laws.
      2. Due Diligence: Conduct thorough due diligence before finalizing any property deal. This includes verifying the title of the property, checking for any encumbrances, and ensuring that the property has clear ownership. Engaging a local lawyer who is familiar with Uttarakhand’s property laws is advisable.
      3. Registration and Documentation: Ensure that all property transactions are duly registered with the local sub-registrar’s office. Proper documentation, including sale deeds, mutation certificates, and tax receipts, is essential for a legally sound purchase.

      Tips for Prospective Buyers
      If you’re considering buying property in Nainital, here are some tips to keep in mind:

      1. Research the Market: Spend time researching the local real estate market to understand property values, trends, and potential areas for investment. Visiting the area in person can provide valuable insights.
      2. Engage a Local Real Estate Agent: Working with a local real estate agent who has in-depth knowledge of Nainital can simplify the buying process. They can help you find the right property, negotiate the best price, and navigate the legal procedures.
      3. Consider Long-term Plans: Think about your long-term goals before making a purchase. Whether you’re buying a property for personal use, rental income, or future resale, having a clear plan will guide your investment decisions.
      4. Assess Infrastructure and Amenities: Consider the availability of essential amenities such as water supply, electricity, and road connectivity when choosing a property. Proximity to markets, schools, and healthcare facilities is also important, especially if you plan to live there permanently.

      Conclusion
      Buying property in Nainital offers a unique opportunity to own a piece of paradise in one of India’s most beautiful hill stations. With its stunning scenery, investment potential, and peaceful lifestyle, Nainital is an attractive destination for real estate investment. By carefully considering the types of properties available, understanding the legal aspects, and following the tips provided, you can make a well-informed decision and enjoy the many benefits of owning property in this charming hill station. Whether it’s a cozy cottage by the lake, a modern villa with mountain views, or a commercial venture, Nainital has something to offer every aspiring property owner.
       

      • 0

    2. To run ChatGPT using LLMs locally, set up an appropriate environment with sufficient computational resources. Install the necessary frameworks like PyTorch or TensorFlow, and use a pre-trained model. For more detailed assistance, our Cheap Assignment Services can guide you through each step.

      • 0