UKH

Just curious TBH - We're all over it where I work, in fact it's now a fundamental part of what we do as a business and I get to use and develop it in real world scenarios all day every day (I'm also incredibly lazy and use it to do a lot of the leg-work that my role entails and stuff just happens  - how cool is that).

Who's using it at work, for research, at home even, what do you think?

S

Scarily good at some stuff, predictably bad at others.

I'm joint chair of the group 'managing' Gen AI in education at my university. I have a particular responsibility for assessment and it's made my life 'interesting' for the last 14 months.

I use it as an ideas generator, I use it to evaluate how it can help or hinder our assessments; I curse it as a major spanner in my work, and see the huge benefits of to support inclusion.

I measure things for clients in return for money.  I make a lot of money by measuring things for my clients.  I don’t use generative AI because I provide a hard measurement service.  I don’t use any other form of AI because I want to provide my clients with a rational, science based pipeline that goes from very complex inputs to hard, quantitative outputs.

My personal view is that current generative AI is a turbo boosted search engine with more associate awareness.  It’s massive flaw is that it gives outputs with no referencing to the inputs and, critically, no quantitative score of how much it’s hallucinated the outputs vs made outputs logically based on inputs.

In terms of universities, IMO generative AI matches my experience of 2:2 or 2:1 level students - blah blah subject specific stuff blah blah but f—k all insight.  This pretty much represents the ends of meritocracy when it comes to degree scores outside of hard STEM subjects.

Hmmm - this one is interesting Wintertree - so the models I work with go through a hell of a lot of training so that they are context aware, we even have off the shelf models for certain industry situations now, and I think that might be the game changer. What you get with ChatGPT for instance or CoPilot are yes essentially semi-sentient search-engines that scrape anything and everything to give you the answer they think you want to get. With the enterprise models the experience is a lot richer, less shit gets spat out, and the models are less prone to bias and hallucination. But you do need data, and you do need extensive an continuous training of the model, it's like having a teenager. They know an awful lot about an awful lot, but specifics tend to be shaky ground when pushed.

A lot of it does come down to prompt engineering though - the response you get is only as insightful as the question you ask, and the data you provide for reference.  Given the fact that I had an intern cheat on a test recently, asking your Gen AI tool to write responses in UK English should be the opening line in any prompt!

 

A bit like the web in the late 90’s; pretty shite, lots of hype and totally irrelevant to our business at the moment. As the devil is in the details it will take a huge human effort to interface what we do with AI, but I guess it’s coming in 10 years (probably less, as things seem to be accelerating). I did get the company AI chatbot to write my Christmas thanks to my team; it wasn’t bad, but it was a bit gushing and wanky, so I wrote my own. 

Interestingly, Bard will not be drawn on the causes of the Israeli-Palestinian conflict. Chatgpt does give an answer but it's pretty basic.

I haven't used it as much, but if you use Bard you can check the sources to some extent.

I like the gpt4 integration in edge, particularly it's ability to read documents you have open. It's a time saver to be able to open a large pdf document and get bing to precis it and I've found it quite accurate. It's also pretty much replaced search for me unless I'm trying to part with my money for something.

One of the main reasons I reach for it now is how poor google search has become over the last couple of years, it drives me mad. An example from yesterday was trying to work out the cost difference in raw material of granite and brick when working out the rebuild cost of a house. Despite various searches all it gave me were places to buy granite work tops over and over. I have this experience with google most days at work, it's becoming infuriating for research purposes but great if you want to find where to buy a pizza. It's interesting that google have recently announced job cuts in their search department. I've worked with seo a little bit over the last 10 years and the idea of a search engine trawling the internet and returning relevant webpages is pretty much gone. LLMs seem to be filling the gap but I miss the days of 100s of pages of organic search results.

Either way, to the OP, it's always open in the corner of my screen at work, but for all the time it saves it probably distracts me more! We use it a bit for writing copy and creating graphics but it's more of an ideas generator and we are very much consumer level users. My attempts at training it on our database of emails and previous year's reports have failed miserably! 

What you describe sounds much more bounded to the point it is a well boxed up semantically flexible enquiry engine which is very useful/powerful but which isn’t really “generative”.  It’s going to answer questions across its knowledge base but isn’t going to make new content in the way generative AI pretends to.

Your post very much put me in mind of Prolog by the way.  It never quite got there…

Each year I have to write a report which is sent to HMRC, basically to claim for a large chunk of my salary in a tax credit for innovation. Writing the report is a right pita as it involves having to pretend that software engineering is hard scientific research in order to fulfil the criteria specified by HMRC, which assume that your innovation is based around that.

I write a glowing report of what we actually did, and then the gatekeeper says no, you didn't address the criteria, and we eventually end up with a load of simplified semi-intelligible waffly b*ll*x that passes muster. I'm really hoping that the new AI tools will basically do this job, with appropriate prompting, next year. It sounds like an ideal task.

Isn't he just describing the same underlying model but with industry specific training data. That always seems to be the great promise of GenAI models whenever I see people touting the benefits in a work context. Just as generative, but "knows what it's talking about" a bit better.

Thats my take.  However…

The quality control of the training data can make a big difference though, as does the use case. I’m interested in what Stack Overflow have announced, where they plan to build an engine trained on their content - which is well moderated and user scored, and provide source references in the outputs.  Boxed up as a well trained, reference supplying semantically capable search engine for a specific domain it could become a useful tool.  In this context and what the other poster described, I’d argue it falls short of being generative AI but marketing departments are all over that term.
 

I guess as soon as it's telling you the answer in it's own text rather than finding you a link that likely contains the answer then that's generative, by definition, even if it's providing a link to the source material from which it "knows" the answer at the same time.

If you want something more like the old Google, you need to consider yandex. I've used it to find pages obfuscated from Google (there are tricks people play to make sure Google search spiders completely misunderstand and can't catalog it, but look normal to humans). There are search operators, and you get results based on pages and page content not advertising sh** .

There is a bit on an elephant in the room discussing it though 

Use it a lot at work. It's kind of fun it to summarise a meeting or write minutes for you. It does save time but not as much as hoped as you absolutely have to read what it's written and correct stuff. Couple of examples:

I asked it to write a document for me based on another document and it made some things up that weren't in the source document so watch out for that! It referenced an industry standard framework (which wasn't mentioned in the source doc) represented by an acronym but then made up completely different words which the acronym stood for...very weird!

Second example is writing code for you. Here it does work really well. It can generate all the scaffolding and boring bits leaving you to focus on the stuff you spend all your time on anyway, namely fiddling about with logic or UI components. So it saves time definitely but proportionally not as much as I thought.

It's great for procrastinators or people who don't like or know how to start a piece of work as it does get you going, but read/check everything!

This has to be true and I have been worrying about it. I think it's inescapable although not many non-STEM academics are willing to say as much. I'm worried that human research skills are going to die out, or become the preserve of a very very few.

As a litigation lawyer I'm horrified - already one reported case of a litigant-in-person innocently presenting to the court the fictitious case-law dreamed up for her by AI, with no idea that the credible-sounding  references were hallucinations.

See my reply to wintergreen above - I seem to recall you're an academic classicist and you are at the sharp end of this - are we doomed?

isn't this what software like visual studio etc has been doing for years? or do you mean something more advanced? example?

I think I’m turning in to a grumpy stick-in-the-mud when it comes to generative vs regurgitative.  The hallucinations are closer to true generation perhaps.  But it’s pointless pedantry from my corner I suppose.

In terms of what it's doing, there's no difference between hallucinations and the rest of the text it generates, as far as I understand it - it's always "guessing" what the right response is, hallucinations are just when it doesn't "guess" right, or when it's "guess" is wrong in context (e.g. when it makes up fake paper references or legal cases, it knows the structure of what a paper reference or legal case reference looks like from having digested a lot of scientific papers and it just fills in the blanks. Sometimes it gets the right correlations and picks out a real paper, perhaps even a relevant one, sometimes it doesn't)

I agree - but when it effectively makes things up it is generating something that didn’t exist before rather than regurgitating understanding.  Which outcome you get is partly happenstance and partly the alignment between the prompts given and the data it’s been trained on.  Mechanistically the same thing happens.  The more I think about the difference the more questions it raises about the origins of human creativity and intuition vs recall.  How much of human intuition is randomness filtered through an understanding of the properties the solution should have, allowing new information to be created?  Can LLMs get to the level of human intuitive creativity in physics or the arts say?  

It doesn't understand anything about geopolitics. It is enormously impressive technology but it is essentially extremely powerful autocorrect, and needs to be understood on that basis. 

If you ask ChatGPT what caused the American Civil War, and it answers "slavery", it does so because that is the most probable answer to that question given its (enormous quantity of) training data - not because it knows anything about history. This is why it "hallucinates", because it's a *Language* model, not a *Knowledge* model.

ChatGPT and Midjourney and the like have caught the popular imagination but for me the really exciting stuff is things like DeepMind's AlphaFold.

Did you mean 'the end of meritocracy'? If so, I'm surprised at that. On the humanities side I can't really see much of a threat, apart from it being a massive timewaster (for which problem there will no doubt soon be versions of Turnitin that purport to help, and which in turn are themselves massive timewasters for those that use them. The human skills involved in bullshit-detecting are worth developing on their own, and part of the point). I find that what gen AI produces is categorically distinct from what I'm looking for, and that what it lacks is so fundamental, despite its superficial similarity, that there's not really any problem in sorting the wheat from the chaff.

That's a very helpful insight, thank you!

It's not that bard gives a poor answer about the causes of that conflict. It won't give an answer at all, its response is to go Google it. The developers have obviously decided that there's too much to go wrong with that particular question!

Can I be alone in not wanting anything to do with so-called AI in the form of LLM queries?

I regard it as pollution of human knowledge. I have used Chat GPT to assist me in learning new technologies for my job of engineering software development and maintenance. It can help a bit over reading the manual but the knowledge gained by me about my subject is more superficial and more easily lost to me.

https://visualstudio.microsoft.com/github-copilot/

I had a similar pondering many years ago, after a UKC discussion on freewill, and the mechanisms by which the brain processes information.

This is a fair point, it eludes to not having to know/learn anything because 'a thing' knows what you need to know and will articulate it for you/even programatically do it for you - so why bother learning anything?

The way I/we look at this is in terms of 'augmentation' in that it's bringing a new level of accessibility to skills and knowledge, and new ways/approaches to looking at and developing 'things'.

Very prosaically, I have two projects on the go that don't use Gen AI to its fullest in the they are not 'creating' anything. Both though would be difficult without it.  One project is to reduce effort in the initial stages of RFP response filtering using GA to look across responses, interpret technical and marketing content, tone and style of writing, mark against a rubric, and then spit out those that are worthy of further review and down voting. The other looks at energy usage patterns and numerous predictive models that inform when and for how long grids swap from one energy source to another, or fall back to storage based on demand - and this also incorporates energy trading so that instantaneous decisions can be made on when to buy and sell capacity. All of this has human oversight though - especially the trading part as traders not only know their markets, they also have 'gut' feelings about things, have side chats with other traders, and these nuances are yet to be built in to the Gen AI models we're trialling.

Also - humour, Gen AI is terrible at sarcasm and at spotting purposefully rubbish content  - I generated a document that was full of errors and spoofs to use as an example of what bad looks like in an RFP response - Gen AI marked it as being acceptable and meeting the higher marking grade of the rubric, even though it's a blatant piss-take.

You could argue however that there is no such thing as free will, and that all our actions are informed one way or another by the inputs we receive from many sources, direct and indirect. In the same way, Gen AI responses are not only it's ability to freely create output/content/opinion based on the prompt you give it, but also the wide, wide, ever more tenuous network of data it has available to it - so it's assumptions, (our assumptions) are never free-form/free-will they are always informed to some extent.

Hence hallucinations and bias - and we are all guilty of misremembering and we are very much in the era of that awful thing the 'alternative fact'.

Yes and no - I have a front-end being developed/coded using parallel teams at the moment, one is Gen AI (horribly labelled Digital Workers) and the other is a real people team. 

The Gen AI team is 60% faster, higher level of code quality, is developing accessibility features in advance of the dev team (who claimed they would look at this as part of future itterations) and developed itself as a whitelable solution without being asked to do so as it defined this as the most efficient way to develop given the nature of the brief. Link this to generating user stories and personas from scratch and you have a very nice production line all the way through to user testing - which is done based on the personas and wild cards.

Although this gets kind of interesting philosophically - I work with retrieve-and-generate systems, ie systems based on the idea of using a store of documents and a search engine to find text that contains the relevant information and then just prompting the LLM to answer the specific question based on the retrieved text. And you think, hey, this makes total sense, and maybe with this sort of approach we won't actually language models to function as both knowledge stores and language models, we can use smaller and cheaper ones that function purely as language models and we'll still be fine. But after a bit of thinking, you realize that actually it's maybe not that easy to separate the two, because parsing and predicting language to a high level essentially involves understanding how words relate to concepts and how those concepts interrelate, and that essentially means having a store of knowledge.

Indeed we are, but we can develop robust processes of logically checking our conclusions to see if they’re wrong.  This is very tied up I think in the mathematical part of our thinking and is much more hard logical than statistical.  I’d imagine any path to useful AGI is going to need to merge some very different types of system together to bring robust consistency checking to LLM style work.

Ha, exactly this, except I managed to escape the chair role, I'm just a regular member of our group!

It's useful for generating images for presentations or reports.

Saves buying them.

And with a bit of tweaking I can normally get pretty close to what I want. When buying images it's just pot luck if what you want exists.

Your requirements must be ambiguity free and really simple. Both teams are getting the same inputs? Does the Gen AI team have a team of human compilers? My experience of CoPilot, etc. is that they'll spit out code that doesn't work in seconds.

Generative AI is creating useful code without coaching? Having seen developers using LLM to build algorithms, some of whom work for the likes of Nvidia and Microsoft, I'm pretty sceptical. They can create useful code but in my experience it takes multiple iterations of refinement involving human input to do so.

I use Bard and ChatGPT and for the use cases I have it is incredible, fast, accurate and whilst I tweak the outcomes, it has been revolutionary for what I need it for.

Of course there's coaching, there's coaching/pairing even when you use people and yes it is iterative like any good rapid prototyping/scalable PoC should be.

That's the thing, with time the coaching becomes less and less as the model learns what it is to design and develop a good UI/UX and the back-office framework that supports it, efficiencies and design innovation happens without intervention.

This seems like a contradiction to me. How is the model going to learn when eventually the only training data it will have are its own outputs? I think there's a reasonable case to be made that the quality of Gen AI output has already peaked. Eventually it will be eating its own shit.

I came across this today about writing fiction and how-to manuals.

https://www.theverge.com/c/23194235/ai-fiction-writing-amazon-kindle-sudowr...

I do think the eating it's own shit thing is a real risk. The most jarring thing for me in the article was about the people who felt they 'needed' to have a book in their name to lend themselves authority in a noisy world, along with the idea of a 'minimum viable book'.

Perhaps drowning in shit is more appropriate.

Coding is distinctly an edge case where the benefits of llms are clear - at the moment.

I've used it a bit for writing code (for myself, not for an employer) and, as others have said, the results are variable. At worst it hallucinated a plausible but completely fictitious algorithm and implementation library that would have been perfect for what I wanted to do. A best it has summarised complex documentation for me, and once suggested a tactic for solving a knotty coding problem that involved combining two different open source code libraries - which quite impressed me.

I'm still not sure how I feel about it tbh. Something tells me the hallucination thing will turn out to be a strength in the long term, but I'm not sure I could say why.

On the flip side, here's Mhairi Aitken from the Turing Institute talking at the RI about the ethical risks and resource consumption of Generative AI technologies.

youtube.com/watch?v=si1jcl7UFqU&

The way we have looked at this is that the model learns both from its own analysis of the 'correctness' of its outputs in comparison to the parallel code written by the dev's plus corrections fed in to the model to correct bias and hallucinations. Over time the model will grow organically, as will its ability to learn from itself, with a commensurate increase in accuracy. The other way we are looking at training the models is reverse engineering, taking existing, long established code, creating a prompt that both analyses and re-writes, and compares new with old.

There's no right or wrong at the moment, just trial and error and everything I've deployed GenAI on for code writing has been PoC and is never any where near anything that might be considered mission critical.

Let's take something more prosaic though, one of the prompts I've written is for a procurement/RFP PoC, I took a scoring rubric, a shitty answer, a good answer, and an exemplar answer, and asked for the two answers to be compared with the exemplar, critiqued, scored, and marked as pass or fail and with recommendations for improvement in both included in the critique. Worked well, referenced not only the RFP and exemplar, but also external data on implementation and architecture. This cut down the analysis time, was fair, (met the explainability rule) and meant that the reviewer could concentrate on those that met the grade more quickly and with better insight in to the technical aspects of the proposals - in essence it cut through the BS.

I think that's a good point, it's just working out where and when and to what level you want the hallucinations (creativity/free-will) to come in to play and do their thing.

Microsoft are using it (well there's a surprise). I was looking at managed identities in Azure today while trying to summon the will to carry on living, and was cheered up by the following snippet:

 "AI-assisted content. This article was partially created with the help of AI. An author reviewed and revised the content as needed. Learn more."

Yes. I want to learn more. However that pointed to a quite reasonable explanation of how they use Gen AI.

The article itself was actually ok too. Bring it on.

Hallucination hints of creativity.

If over time the balance morphs from useless to useful it changes from hallucination to thinking intelligently, or being indistinguishable from intelligent thought. 

I don't think it really is. All the model is doing is a guessing game as to what the "right" text to respond with is, and a hallucination is when it gets it wrong. It isn't "making stuff up" any differently than usual, it doesn't have a concept of what the right answer is, just what the most likely answer is.

The analogy I've seen in a few seminars/learning sessions is that the models are doing the same thing as the predictive text on your phone, just at an enormously more complex scale. 

No you’re not alone. I haven’t gone near it. If I am producing something I want to understand where it came from and be sure that it represents my thinking.

Humans can do fuzzy statistical stuff and hard logical stuff.  LLMs only do the former.

Yeah, for some value of more complex. Like, you can produce a semi-useful predictive text that knows that "are you climbing this _" is normally followed by "weekend" simply because you keep using the phrase "climbing this weekend" and not "climbing this tree" or "climbing this pineapple". OTOH a LLM can demonstrably handle more that - if you say "my brother's a gritstone fanatic, so when his software job went remote he moved to _" you'd expect it to be able to identify that Sheffield is a much more likely continuation than London, which can't be accounted for by just having seen lots of "he moved to Sheffield" before, but requires it to pick out that a) his enthusiasm for grit (and not his software job) is the relevant bit of information to where he moved and b) that a gritstone fanatic is more likely to want to move to Sheffield than London. Which is to say that it needs to be able to parse the sentence structure and apply some world-knowledge.

In other words, yeah, it is initially trained as a really good predictive text, but predicting what humans are going to type essentially means predicting what humans think, and as you get better at doing that it essentially means developing progressively better ways to approximate to human thinking.

Agree on hallucination, btw.

I think we mostly agree, but I am not sure I would say "think" above, more "have typed" - in the sense that the model only knows that Sheffield is a word that should be associated with gritstone because people have written a lot about Sheffield and the grit in the material it has ingested. 

It needs to be able to understand far more in terms of language structure than a simple predictive text model, of course, which I hoped I'd covered off in "enormously more complex". 

That's all very well, however that is not at all how the current systems work. In order to get one of those, you may need a completely different approach.

It's intriguing to speculate whether an existing LLM may be able to derive such a system though; you would have to assume that the way humans manipulate language could be modelled by some shape of network. Possibly WT is on the right track suggesting one or more ways to verify a proposition.

I imagine some kind of conceptual store, for example where "gritstone" might exist in multiple places including geology and climbing, with some way to measure conceptual similarity. Google has been doing this kind of thing for a very long time with search technology; probably using the statistical approach, but who knows?

Also Apple and/or IBM research teams had a crack at it a long time ago, but it was all very hush-hush and most probably nothing came of it - as far as we know. Now where's my tinfoil?

Is it the word "understand" that you're disagreeing with here? If so then yes, in hindsight that is a lack of precision that seems to have been a regrettable casualty of trying to multitask this with being at work.

It can draw correlations/linkages that hold up across far more complex linguistic structures than have been possible previously such that it can often produce sensible answers to quite complex questions.

Yes, the results are astonishing, and I don't pretend to know any of the intricacies. It seems mind -boggling that it can produce what it does based on probability. Perhaps it can.

For the thread in general, a wonderful example of how generative AI can produce utter crap, and how naive people who should know better can be about accepting it.

This is an AI that generates images rather than text, but AI-generated text submitted by students for assessments is just as hilariously/disturbingly bad.

https://scienceintegritydigest.com/2024/02/15/the-rat-with-the-big-balls-an...

This paper has now been retracted.

On the flip side, you have OpenAI seemingly taking a giant leap forward in generating HD videos from text prompts:

https://arstechnica.com/information-technology/2024/02/openai-collapses-med...

True - the real problem in my example is the (lack of) responsible use...

Intuitively it doesn't seem that surprising - you're taking a thing which is basically a very general family of information processors and then using hundreds of billions of training examples to try to get one that's good at a task, via an explicit feedback mechanism for improving their processing structure based on their mistakes.

Given that the task is basically about language and world-knowledge, it's kind of understandable that it develops relatively powerful and general capabilities for doing other tasks based around language and world knowledge. I mean you could argue it's more surprising that humans managed it in only a few billion years given that our training objective was just to create as many copies of our genetic material as possible and we didn't have gradient descent algorithms to help us...