NVIDIA Chat with RTX is available for download
Artificial Intelligence assistants like ChatGPT are highly useful, though they require an internet connection for access. While local deployment of Large Language Models (LLMs) is feasible, it often presents considerable complications. Recently, NVIDIA introduced a more convenient method for the local deployment of AI assistants, dubbed Chat with RTX.
NVIDIA describes Chat with RTX in its official blog as a technical demonstration application that allows users to customize a chatbot with their content. This application integrates retrieval-augmented generation, TensorRT-LLM, and NVIDIA RTX acceleration technologies. Users can connect local files from their PCs as datasets to open-source LLMs such as Mistral or Llama 2, enabling them to query the AI for answers and relevant files without manual searching. NVIDIA notes that Chat with RTX supports multiple file types, including .txt, .pdf, .doc/.docx, and .xml. Users simply need to provide the folder path, and the AI will load them into its library within seconds.
Furthermore, NVIDIA claims that the AI can understand and extract content from YouTube videos (and possibly more websites), requiring only the addition of a YouTube video or playlist URL. Users can ask the AI to summarize video content, for example. Although this feature necessitates internet access, the AI’s processing is entirely local.
As for system requirements, users need a graphics card with at least 8GB of VRAM from the RTX 30 or 40 series (encompassing both Ampere and Ada Lovelace generations), over 16GB of RAM, a Windows 11 operating system, and drivers version 535.11 or newer. The VRAM requirement is not exceptionally high. Regarding storage, users must download a 35.1GB file. Interested users meeting these requirements can directly download the application via the provided link.
