Large Language models

[leothetechguy]

I'm fascinated by all of the possibilities (translation, logical reasoning, smart data processing, being able to write and analyse code), but also concerned on how to ethically source the huge amount of training data.

And since the Architecture of these models is called a transformer (and also operating in that way), AI technically circumvents the copyright laws of the US under Fair Use, which allows transformative works.

A key consideration in later fair use cases is the extent to which the use is transformative
https://en.wikipedia.org/wiki/Fair_use

Would love to hear some of your thoughts about them.

[Moonchild]

My opinion on the matter is that LLMs (or any AI model for that matter) should only be trained on data that is opt-in/has explicit permission to be used for training.

[Blacklab]

If I understand Large Language Model (LLM) AI correctly they are simply computing the most likely next word on a vast scale.

This by means of 'learning' the probabilities of the 'next word' from huge databases of language, mostly sourced (scraped) from what is available from online communications including the large social media platforms (stolen or otherwise).

Thus LLM AI is not in any sense 'intelligent' or even vaguely trustworthy in the basic sense of giving factually correct or logical answers... it can and will 'lie' blatantly as long as the next word fits the generated algorithm, as is already apparent from distorted, untrue, but plausible biographies, etc. It is a form or Russian Roulette played on a vast scale with words... and thus should not be relied upon for anything. Other types of AI may do things differently.

As with all learning the old adage 'rubbish in = rubbish out' applies, and probably in spades regarding the databases used to train LLMs.

Unlikely that internet social media chatter would score very highly as a good source of learning... and now that LLM are roaming free on the internet they will already be feeding on themselves (i.e. sourcing 'learning' from databases increasingly infected with the output of other AI).

Then you are just looking at the output from a feedback loop of truly monumental proportions... not exactly 'standing on the shoulders of giants'... rather a hysteresis loop gone mad?

[vannilla]

LLMs are meant to generate sentences that read like they are written by a human, or, anyway, that are grammatically correct.
The fact that they are advertised as a way to get answers to your questions is a scam born from their source material.
LLMs are just a gear that need more gears to actually provide a functional machine.
A simple example I've seen in the wild that somehow works: you have some text in a foreign language, you pass it to a translator, the translator gives the general meaning to the LLM and it creates a grammatically correct output. While the meaning depends entirely on the accuracy of the translator, at least you don't get the obvious errors like wrong pronouns mid-sentence.
Due to their role, LLMs could technically be sourced entirely in-house simply by writing a bunch of sentences in different styles (e.g. using a certain word instead of another), but as the "Large" in the name suggests, they need a lot of data and scraping the web or whatever is a lot cheaper.

[RealityRipple]

Claude Shannon's "A Mathematical Theory of Communication" (link here) posited this concept in 1948. As a starting point. It's literally like someone read the first 7 pages and thought "that's enough to scam the world with".

[leothetechguy]

If I understand Large Language Model (LLM) AI correctly they are simply computing the most likely next word on a vast scale.

While True, I think that's a massive oversimplification. The best introduction to Transformer models are these two blog posts imo:

https://jaykmody.com/blog/gpt-from-scratch/
https://jaykmody.com/blog/attention-intuition/

There is a neural network that is deciding which information is similar to each other, and that stores it in a big vector space, but does it really just try to group information which looks similar? Or can it learn deeper connections, similar to the connections we form in our brains for logical reasoning?

[Blacklab]

The best introduction to Transformer models are these two blog posts imo:

https://jaykmody.com/blog/gpt-from-scratch/
https://jaykmody.com/blog/attention-intuition/

Thank you for those links... although I suspect the highly technical code-based nature of those two blogs will likely baffle rather than illuminate for those of us not well schooled in coding or the deeper structure of artificial intelligence models.

At the other end of the 'understanding AI' spectrum Vodaphone's August Newsletter has an article 'Everything you need to know about AI... How does ChatGPT actually work?'... with some nice examples of LLM AI image outputs for anyone who has avoided using the technology so far...see: https://www.vodafone.co.uk/newscentre/f ... ink-it-is/

IMO the immediate and overwhelming problem with the sudden emergence of AI online is that such outputs are not clearly marked, notified, or differentiated to the user. Thus, users are unable to quickly (if at all) determine what level of trust or confidence to place in any particular piece of digital material, be it word, image, or code-based. IMO this lack of any clear 'AI generated' attribution will cause the catastrophic undermining of trust in online information of all sorts... with obvious negative effects on everything from online shopping to news reports and political discourse.