PABLOF7z

f7z.io

Magical Other Stuff Maximalist

April 23 2024 · Edited Jun 10, 2024

NDK 2.8: "Fast and steady"

NDK 2.8 has been released, coming with a plethora of performance improvements, threading capabilities and many many other things. TL;DR: Optimistic signature-verification Threaded signature-verification Threads, threads, threads! s/o Dexie cache adapter becomes massively more efficient Onwards 🫡

by PABLOF7z

February 1 2024 · Edited May 28, 2024

Shipping Shipyard DVM

Shipyard DVM is here!

by PABLOF7z

February 28 2024

We can. But should we?

Will importing Wikipedia destroy Wikifreedia, the nostr-based wikipedia alternative?

65 ## profound impact large language models are about to have. If DQN and AlphaGo were the early signs of something lapping at the shore, ChatGPT and LLMs are the first signs of the wave beginning to crash around us. In 1996, thirty-six million people used the internet; this year it will be well over five billion. That's the kind of trajectory we should expect for these tools, only much faster. Over the next few years, I believe, Al will become as ubiquitous as the internet itself: just as available, and yet even more consequential. ## BRAIN-SCALE MODELS The Al systems I'm describing operate on an immense scale. Here's an example. Much of Al's progress during the mid-zotos was powered by the ef fectiveness of "supervised" deep learning. Here Al models learn from carefully hand-labeled data. Quite often the quality of the Al's predictions depends on the quality of the labels in the training data. However, a key ingredient of the LLM revolution is that for the first time very large models could be trained directly on raw, messy, real-world data, without the need for carefully curated and human-labeled data sets. As a result almost all textual data on the web became useful. The more the better. Today's LLMs are trained on trillions of words. Imag ine digesting Wikipedia wholesale, consuming all the subtitles and comments on YouTube, reading millions of legal contracts, tens of millions of emails, and hundreds of thousands of books. This kind of vast, almost instantaneous consumption of information is not just dif ficult to comprehend; it's truly alien. Pause here for a moment. Consider the unfathomable number of words that these models consume during training, If we assume that the average person can read about two hundred words per minute, in an eighty-year lifetime that would be about eight billion words, assuming they did absolutely nothing else twenty-four hours per day. More realistically, the average American reads a book for about fifteen minutes per day, which over the year amounts to reading about a million

E # sed in electrical grids and ## THE TECHNOLOGY OF INTELLIGENCE # sources at a time of grow- 63 ## about what comes next. Their work laid the foundation for what has # st how to write emails or been nothing short of a revolution in the field of large language modrite stories, diagnose rare els (LLMs)-including ChatGPT. ## LLMs take advantage of the fact that language data comes in a se- # e change. They feature in quential order. Each unit of information is in some way related to data # homes. You already inter- earlier in a series. The model reads very large numbers of sentences, # : many more, and almost learns an abstract representation of the information contained within # fficient, faster, more use- them, and then, based on this, generates a prediction about what should come next. The challenge lies in designing an algorithm that "knows where to look" for signals in a given sentence. What are the key words, the most salient elements of a sentence, and how do they ## HING: relate to one another? In Al this notion is commonly referred to as "attention." ## ¡E MODELS When a large language model ingests a sentence, it constructs what can be thought of as an "attention map." It first organizes commonly guage seemed too comoccurring groups of letters or punctuation into "tokens," something Then, in November 2022, like syllables, but really just chunks of frequently occurring letters latGPT. Within a week it making it easier for the model to process the information. It's worth alked about in rapturous noting that humans do this with words of course, but the model doesn't ght eclipse Google Search use our vocabulary. Instead, it creates a new vocabulary of common tokens that helps it spot patterns across billions and billions of docu- But it is so much more ments. In the attention map, every token bears some relationship to every token before it, and for a given input sentence the strength of ## ad previously been made this relationship describes something about the importance of that aneously in fluent prose. token in the sentence. In effect, the LLM learns which words to pay # siness plan in the style of attention to. ## does so in seconds. Ask it So if you take the sentence "There is going to be a fairly major # ting manual, or a Python storm tomorrow in Brazil," the model would likely create tokens for the letters "the" in the word "there" and "ing" in the word "going," # ent is that we look at the since they commonly occur in other words. When parsing the full sen- # ure. In this sense intelli- tence, it would learn that "storm," "tomorrow," and "Brazil" are the # rate a range of plausible key features, inferring that Brazil is a place, that a storm will be happening in the future, and so on. Based on this, it then suggests which # lay unfold and then base tokens should come next in the sequence, what output logically fol- # 2017 a small group of re- Lows the input. In other words, it autocompletes what might come # ver version of this prob- next. # on the most important and efficient predictions

Today we're visiting the island in front of ours. Telendos. No roads. Iirc about 15 ppl live here y...

#olas365

Around the corner (Yes, this is a test) #olas365

Things are insane right now, and maybe I’m too.

Most important, the public needs education on Al so they can pressure for an aligned future as informed citizens. To- day's decisions about how Al reflects human values and en- hances human potential ...

Over the last 5 years, we have watched as Ai has taken over the world, it feels like everyday there’s something new, and it’s hard to keep track of if you aren’t actively monitoring it, but it’s real.