![Matt Williams](/img/default-banner.jpg)
- 296
- 1 543 968
Matt Williams
United States
Приєднався 23 сер 2006
I was a founding maintainer of Ollama, the first evangelist at Datadog, and a former organizer of DevOps Days Seattle, DevOps Days Boston, and Serverless Days Boston. In my day job, I tour the country speaking at conferences and writing about the company I work at. But I am also passionate about gadgets. Some folks don't think that word is appropriate, but it totally fits here. If you meet me in person you will see me light up when I can share with you my latest gadget, tool, or utility. And this is where I get to share that with all of you. You can find more about me at my website (technovangelist.com) or on Twitter ( technovangelist)
AI News for 2024-08-17
Today is Saturday August 17, 2024
## Articles covered:
### Techcrunch overview of Copilot
techcrunch.com/2024/08/17/microsoft-copilot-everything-you-need-to-know-about-microsofts-ai/
### ColPALI model
medium.com/gptalk/colpali-the-ai-that-sees-documents-like-you-do-93ede84c052e
### Russian interference in elections not so good this year
www.theguardian.com/technology/article/2024/aug/15/russia-us-election-interference-ai-meta-facebook-instagram
### AI Revolutionizing Fire Fighting
www.theguardian.com/world/article/2024/aug/15/how-ai-is-revolutionising-how-firefighters-tackle-blazes-and-saving-lives
### How to get from HS math to AI
www.justinmath.com/how-to-get-from-high-school-math-to-cutting-edge-ml-ai/
### A better search engine algorithm
www.mixedbread.ai/blog/intro-bmx
**Simplified Analogy**
Imagine searching for books in a library:
• **BM25:** Looks at how many times specific words from your search appear in each book and gives more importance to rare words.
• **BMX:** Not only does this but also looks at how those words fit together in the context of your entire search. It also gives more importance to words that provide more unique information.
In summary, BMX takes the good parts of BM25 and makes them better by considering the overall context and importance of terms, leading to more accurate and relevant search results.
### New Satellite with AI on board
www.esa.int/Applications/Observing_the_Earth/Phsat-2/New_satellite_demonstrates_the_power_of_AI_for_Earth_observation
Be sure to sign up to my monthly newsletter at technovangelist.substack.com/subscribe
I have a Patreon at patreon.com/technovangelist
You can find the Technovangelist discord at: discord.gg/uS4gJMCRH2
The Ollama discord is at discord.gg/ollama
(they have a pretty url because they are paying at least $100 per month for Discord. You help get more viewers to this channel and I can afford that too.)
Chapters
00:00 Microsoft's Copilot and its Variations
03:33 AI Tools for Interpreting PDF Documents
08:16 Russian AI Efforts to influence the US Election
10:43 AI in Firefighting
11:50 Satellite Imagery and Onboard AI Processing
13:41 How can LLM's be used in Cyber Security
20:27 BM25 vs BMX
24:14 Learning Math for AI and ML
## Articles covered:
### Techcrunch overview of Copilot
techcrunch.com/2024/08/17/microsoft-copilot-everything-you-need-to-know-about-microsofts-ai/
### ColPALI model
medium.com/gptalk/colpali-the-ai-that-sees-documents-like-you-do-93ede84c052e
### Russian interference in elections not so good this year
www.theguardian.com/technology/article/2024/aug/15/russia-us-election-interference-ai-meta-facebook-instagram
### AI Revolutionizing Fire Fighting
www.theguardian.com/world/article/2024/aug/15/how-ai-is-revolutionising-how-firefighters-tackle-blazes-and-saving-lives
### How to get from HS math to AI
www.justinmath.com/how-to-get-from-high-school-math-to-cutting-edge-ml-ai/
### A better search engine algorithm
www.mixedbread.ai/blog/intro-bmx
**Simplified Analogy**
Imagine searching for books in a library:
• **BM25:** Looks at how many times specific words from your search appear in each book and gives more importance to rare words.
• **BMX:** Not only does this but also looks at how those words fit together in the context of your entire search. It also gives more importance to words that provide more unique information.
In summary, BMX takes the good parts of BM25 and makes them better by considering the overall context and importance of terms, leading to more accurate and relevant search results.
### New Satellite with AI on board
www.esa.int/Applications/Observing_the_Earth/Phsat-2/New_satellite_demonstrates_the_power_of_AI_for_Earth_observation
Be sure to sign up to my monthly newsletter at technovangelist.substack.com/subscribe
I have a Patreon at patreon.com/technovangelist
You can find the Technovangelist discord at: discord.gg/uS4gJMCRH2
The Ollama discord is at discord.gg/ollama
(they have a pretty url because they are paying at least $100 per month for Discord. You help get more viewers to this channel and I can afford that too.)
Chapters
00:00 Microsoft's Copilot and its Variations
03:33 AI Tools for Interpreting PDF Documents
08:16 Russian AI Efforts to influence the US Election
10:43 AI in Firefighting
11:50 Satellite Imagery and Onboard AI Processing
13:41 How can LLM's be used in Cyber Security
20:27 BM25 vs BMX
24:14 Learning Math for AI and ML
Переглядів: 666
Відео
AI News for 08/16/2024
Переглядів 5012 години тому
I'm starting to cover all the AI News from the last 24 hours. This was the stream from earlier today. Here are the articles I covered: old.reddit.com/r/LegalAdviceUK/comments/1etko9h/family_poisoned_after_using_aigenerated_mushroom/ nousresearch.com/freedom-at-the-frontier-hermes-3/ nousresearch.com/hermes3/ nousresearch.com/wp-content/uploads/2024/08/Hermes-3-Technical-Report.pdf crfm.stanford...
Can the Ollama API be slower than the CLI
Переглядів 1,1 тис.4 години тому
In this video, we tackle a common question: Why does the Ollama API seem slower than the CLI, even though they perform at the same speed? Using the example of answering "What is a black hole?", we demonstrate and explain the nuances behind this perception. Be sure to sign up to my monthly newsletter at technovangelist.substack.com/subscribe I have a Patreon at patreon.com/technovangelist You ca...
Technovangelist looks at the news - August 15 2024
Переглядів 5824 години тому
Let's look at the news of Thursday August 15 2024 I looked at the following items: arc.net/e/52DEF3A7-2488-4490-A73E-ACBD26344417 techcrunch.com/2024/08/15/coderabbit-raises-16m-to-bring-ai-to-code-reviews/?guccounter=1 coderabbit.ai/ docs.double.bot/introduction martinfowler.com/articles/exploring-gen-ai.html bloop.ai/ supermaven.com/ mindtown.ai/app ipkitten.blogspot.com/2024/08/openais-grant...
Matt and Ryan have a chat: The StackOverflow Dev Report - edit - from August 6, 2024
Переглядів 727 годин тому
Summary The conversation covers various topics including accents, KubeCon, the Stack Overflow developer survey, learning to code, and years of coding experience. They discuss the popularity of different programming languages, the trustworthiness of AI tools, and the preferred resources for learning to code. They also touch on the role of developer advocates and product managers. The conversatio...
Ryan and Matt have a Chat - edit
Переглядів 1409 годин тому
The conversation starts with technical difficulties and small talk about exercise. The main theme of the conversation is the discussion of browser options and the potential impact of a court case involving Google's search engine dominance. The hosts explore alternative search engines and the idea of a multi-search search engine. They also mention the possibility of hosting AI-based search engin...
The Ollama Course - Using the CLI
Переглядів 2,5 тис.9 годин тому
Welcome back to the Ollama course! In this video, we dive deep into the command line interface (CLI) of Ollama, exploring all the powerful options and commands available. Whether you're a beginner or looking to enhance your skills, this free course on UA-cam will guide you to become an Ollama Pro. 🔍 What You'll Learn: 1. Overview of Ollama CLI Commands: Learn how to list all commands using olla...
Docker Networking Made Simple: Connecting Containers Like a Pro
Переглядів 4,2 тис.21 годину тому
Docker Networking Made Simple: Connecting Containers Like a Pro
Ollama Course - 3 - How to use the Ollama.com site to Find Models
Переглядів 3,2 тис.День тому
Ollama Course - 3 - How to use the Ollama.com site to Find Models
Another Function Calling video? Yup...
Переглядів 7 тис.14 днів тому
Another Function Calling video? Yup...
Installing Ollama - #2 of the Free Ollama Course
Переглядів 2,7 тис.14 днів тому
Installing Ollama - #2 of the Free Ollama Course
Is Dify the easiest way to build AI Applications?
Переглядів 15 тис.21 день тому
Is Dify the easiest way to build AI Applications?
SuperWhisper is my Scriptwriting Super Power
Переглядів 7 тис.28 днів тому
SuperWhisper is my Scriptwriting Super Power
Welcome to the Technovangelist Channel - 2024 Update
Переглядів 1,2 тис.Місяць тому
Welcome to the Technovangelist Channel - 2024 Update
How does function calling with tools really work?
Переглядів 11 тис.Місяць тому
How does function calling with tools really work?
Technovangelist Goes Live - 2024-06-30
Переглядів 700Місяць тому
Technovangelist Goes Live - 2024-06-30
Two small fixes that strengthen Ollama's lead in Desktop AI
Переглядів 15 тис.Місяць тому
Two small fixes that strengthen Ollama's lead in Desktop AI
Challenges with Adobe Subscriptions and more with Matt and Ryan
Переглядів 270Місяць тому
Challenges with Adobe Subscriptions and more with Matt and Ryan
My favorite way to run Ollama: Gollama
Переглядів 17 тис.2 місяці тому
My favorite way to run Ollama: Gollama
The Matt and Ryan Chat on June 4 - Cleaned Up
Переглядів 3002 місяці тому
The Matt and Ryan Chat on June 4 - Cleaned Up
how to reliably get json out of ollama. Just a rough demo for a discord user
Переглядів 3,7 тис.2 місяці тому
how to reliably get json out of ollama. Just a rough demo for a discord user
Have You Picked the Wrong AI Agent Framework?
Переглядів 59 тис.2 місяці тому
Have You Picked the Wrong AI Agent Framework?
Popularity doesn't always mean Great, But Pretty Good is Possible
Переглядів 7 тис.3 місяці тому
Popularity doesn't always mean Great, But Pretty Good is Possible
I think it looks good, but I wasn't watching the stream.
In chat completion is there something like chat request with a pdf file?
Matt, your subtitles are greeting in the way of UA-cam subtitles
I didn’t notice them. Next time I’ll disable
Hey Matt. What's your thoughts on the snake eating it's tail thing going on with LLM models and training now?
I thought I was doing well until you went from cd dify to cd docker, then you completely lost me as my cmd line doesn't recognise the eza -ahl, any thoughts? Thanks
Just change directories into dify then docker
Thank you for these videos. They're really awesome. The format's great - it's casual, informative, relaxing. Also, hello from Europe!
Oh god, there is nothing more annoying that forced color jumping ticktock bullshit subtitles! Unsubscribe!
I didn’t even notice they were there
Nice show!
There is only one viable ocr program and it sucks! I can only get OCR to work by using Ai.
There are so many ocr solutions and no model comes close to what they do.
Nothing much has improved in the tech industry 20 years but it’s been that good for that long
Can you explain why the Windows Executable is significantly slower than when running in WSL? I also found that when running Ollama on Docker with WSL2 backend is faster than just running in WSL. Why!?
Running on windows native in most cases is 10-15% faster than using wsl. If it’s not there is something wrong with the install
Hey Matt don't you have the feeling to be Professor Gerald Lambeau asking Will Hunting to do simple maths ? I know I do :) This question is for boths Matts (@technovangelist and @matthew_berman)
Great job Matt, I enjoyed the live stream.
👌
As usual, wonderful video, Matt. This is exactly the kind of question that I make to myself when learning ollama. And here’s the answer on a silver tray.
Keep up the great myth busting.
So I made a Discord bot that streams from the Ollama API to Discord. Discord has an API rate limit of like 3 requests per second, so I had to buffer all the stream payloads in memory and dump them in chunks to Discord. I did this in NodeRed if anyone wants the code. I had to implement the HTTP client again from the TCP layer to support streaming, but have wrapped it up nicely into a single function node. I should also mention that it supports multiple conversations concurrently (without mixing up the streams).
If you are getting rate limited at 3/s your code is probably doing something wrong. Discord allows 50 requests/sec. I guess unless you have lots of bots.
@@technovangelist This is the only bot I have on my test server, lol. It's either discord or the nodered plugin for Discord. It's definitely around 3/s. It's possible it's because it's an unapproved bot that I just use for testing and the rate limit is lifted when your bot is approved (This could be to help prevent abuse).
Can I train model via chatting with the model? how to do it
Not sure if my original comment posted or not. Either way, Mervin Praison has a neat little add-on for PraisonAI which makes the training process a bit less painful.
Great educational content. Bravo! 🎉🎉🎉
Nice spin on things.
When accessing Ollama via the API, can we set keep alive to forever? If yes would it prevent the API from later switching models?
You can set it to -1 which will keep it in memory until you run out and change models.
Can ollama run on Gpu instead of Cpu? Also how do you incoporate ollama chat you build into website? Thanks for sharing good content🎉
If there is a GPU, it will always use that instead of the CPU. It's kinda hard to not make it use the GPU.
Ah - that discord... says invite expired ! ?
there is a different one in the description
discord.gg/uS4gJMCRH2
Good day, Matt, hopefully, this is my last question about the Private GPT installation. My laptop has arrived. I have installed an M.2 2T primary drive and a secondary 2T SSD. Q: After installing Ollama, Docker, and WebUI, can the models be stored (directed) to the secondary SSD to preserve space on the primary M.2 system SSD? If so, when do I pick where to store the models during their installation?
Local!
hello, any idea how to set keep_alive when running the windows exe ?
I have a noob question. If anybody can upload a model on Ollama, is it possible for a malicious user to upload malware disguised as a model? And are there measures to prevent such a scenario.
There would be no way to do that.
Can I generate an installable app to deploy it in own infra? Which code language use it behind? Can I have access to code generated to custom behaviour?
For that you would need to build something yourself. This is for web based apps that you want help building
Hey Matt, I was really excited for this video, then only to realize theres no mention of Ollama in it!?!? I have a project on the go where I'm trying to build a multi container app using docker-compose where the containers are backend- fastAPI, frontend- Nextjs and llmServer- Ollama. I'm running into problems having the backend connect to the Ollama server ... I get the dreaded [Errno 111] Connection refused
This was 100% about using tools with ollama and docker.
Ahh. I see how you can think that. But I wouldn’t use ollama in docker anyway. This is just about the UIs.
@@technovangelist wow, thanks for responding Matt, much respect for what you're doing! considering your comment "... I wouldn’t use ollama in docker ..." might I be so bold as to ask... if you were me, and you needed to host this app on Azure (which I do) how would you go about hosting Ollama?
Got it. That makes sense. Docker on a host vs docker on localhost can be different. If you are running their container service rather than an instance then that makes sense. Have you had success with getting access to a real gpu? Last time I tried I could only get their generic named cards and not a real amd or nvidia card.
@@technovangelist hi again Matt and thanks for the continuing engagement! to answer your question, Yes we have had success in getting a vm with a gpu, there are some "N series" options on Azure for us mere mortal, e.g. NV4as_v4 which is a 4 core 14gb server with an AMD gpu (this is the smallest of 3 in this series and costs $170p/m). I've stood up an ollama server with one of these and I can test connection to it over the internet successfully, in my app I have the baseURL of the ollama server setup as an env, so I can swap it out... but when I do I get connection issues :( interestingly yesterday I also setup a serverless endpoint on Azure for a llama2 model and ran into the same problems, so this means the issue might be totally unrelated to ollama!?!
Any tutorial on model creating from custom data,like pdf s? Like for companies?
yes please ive been looking for this. if you find anything please share, any help is appreciated
Please share the link of the video for reducing the model size for specific tasks, example, only weather, is wouldn't need the hole context for this
You would be able to fine tune for that but it wouldn"t reduce the size. Reducing the size would be a very expensive process.
1973
Man. I love it. I already subscribe to it. Something I really will be crazy to know is how to story my ollama local models in an external hard drive in Mac. As you know macs doesn't have much space. So, i bought a special hard drive that runs at 40G/sec to have models and other stuff and I will love to have the models in there than in my internal hard drive. Thanks for the great content and explanations.
They don't have much space?Sure they do. Mine has 4TB inside. But you can use the OLLAMA_MODELS environment variable to start storing them elsewhere.
@@technovangelist Would you plan to do a video about it?
Matt, I do not see COBOL (of any flavor) especially for PCs; e.g., MicroCOBOL and or GnuCOBOL 😢😢😢 Which Ai Code Assistant can GnuCOBOL? Is LaTeX not considered a programming language? Or is LaTeX considered native to all Ai entities?
My only suggestion is to try it. Try each model to see what supports it.
@@technovangelist it’s good to see ADA on that list. I had three RFC adopted in the early 1980s and it was my primary language at NASA in the early STS project.
Wonderful video, Matt. Thanks so much for sharing this.
I love your videos! Your explanations are amazing, thank you!
Thank you!
🙏🙏🙏Please how to add vision model MiniCPM-V 2.6 to ollama? openbmb/MiniCPM-V-2_6-gguf
I'm really enjoying this series. Thanks.
what location to run that download hugging face model command? and where does it download to? same location as the others wheres that?
any1 wana swap code for tokens?
I am very interested in FABRIC, and I will probably get into a serious use of it. I think it would be very useful if you create a video taking one or two of the patterns that may not be configured to your exact use case and demonstrate how you would modify them for your use case. This type of use case from an experienced Generative AI Aficionado can be very helpful for more novice users. Thanks for all your videos. I watched a lot of them and I will watch a lot more.
is there a way i can fetch the whole response in a single response object
Sure. Set streaming to false
"If, for example, I have more than one model downloaded, and one is chat, another is multimodal, and another generates images, can I make it so that Ollama chooses which model to use based on a prompt, or does it by default use the one you've chosen with the `ollama run` command?"
It doesn’t do that. But you could build an app that does that.
@@technovangelist ok . 100 Thanks
Hey Matt! Off topic comment but I guess I'm feeding the ol' UA-cam algorithm anyway! I haven't watched your entire backlog so apologies if you've already covered this, but I'd love to see some content / videos on the following topics: 1. How can you use ollama in a production environment. Topics around infrastructure, reasonable techniques (e.g. handing off processing to async jobs when possible), cost, etc. I'm not sure how common this use case is but I am evaluating using something like llama 3.1 to help summarize some potentially very large text files and weighing cost differences between using something turnkey like openai's APIs vs figuring out hosting myself (well my company. There seems to be a lot less on production hardening some of these open source models (or I just haven't been paying attention!) 2. A "state of the union" high level overview of the options available to software developer new to using AI. This you have covered in a lot more detail in various forms, but an overview of what tools are actually at a persons disposal in terms of trying to use AI to solve some problem. When I first started looking at this stuff I thought the only options I had were buying a bunch of super computers to train models and learning a lot about doing matrix multiplication. But we have RAG, we have "fine tuning", we have modifying system prompts... a sort of high level overview of what a layperson can do, and perhaps where reasonable off-ramps for more advance use cases are would be super helpful (i.e. when do I need to brunch up on my linear algebra? :)) Thanks for your work!
Outstanding video. As a complete newbie this was intriguing, well explained and engaging. New subber and bell set
Excellent content Matt! Congrats! Keep on going.