Category: Technology

TL:DR – A number of people were reaching Sciencebase searching for the phrase ChatGPT beam search so I asked ChatGPT itself to give me a definition. I then asked it to create a glossary of other common terms associated with Generative pre-trained transformers, GPTs.

ChatGPT, the language model, is essentially an incredibly sophisticated autocomplete program that responds to text prompts by predicting what might come next. It’s worth noting that it’s commonly referred to as artificial intelligence, AI. It is not AI. Here’s its response to my prompt asking it for a definition of ChatGPT beam search:

Beam search is a search algorithm used in Natural Language Processing (NLP) to find the most likely sequence of words or phrases that represent the best possible solution, given a set of choices.

In simpler terms, when trying to predict the next word or sequence of words in a sentence, there are often multiple possibilities that could be correct. The beam search algorithm helps to narrow down the number of options and find the most likely option based on the context and probability of each word or phrase.

The “beam” in beam search refers to the number of possible options that the algorithm keeps in consideration at each step of the search. By limiting the number of options to a certain “beam width”, the algorithm can efficiently explore the search space and find the best solution more quickly than exhaustively considering all possible options.

I then asked it to generate a glossary of the most important terms in discussions of ChatGPT:

Generative pre-trained transformer, GPT: GPT is a type of language model developed by OpenAI that uses deep learning techniques to generate natural language text. The GPT model is based on a transformer architecture, which was introduced in a paper by Vaswani et al. in 2017, pre-training involves exposing the system to a large amount of written text so that it “learns” the meaning and context of different words and phrases.

Natural Language Processing (NLP): A field of study that focuses on teaching computers to understand, analyze, and generate human language.

Neural Networks: A type of machine learning model that is inspired by the structure and function of the human brain.

Recurrent Neural Networks (RNN): A type of neural network that is specifically designed to handle sequential data, such as text.

Long Short-Term Memory (LSTM): A type of RNN that is designed to better handle long-term dependencies in sequential data.

Attention Mechanism: A technique used in NLP that allows models to focus on specific parts of the input sequence when making predictions.

Transformer: A type of neural network architecture that uses self-attention mechanisms to process input sequences in parallel.

BERT: Bidirectional Encoder Representations from Transformers. A transformer-based neural network architecture that is designed to better understand the context and meaning of words in a sentence by processing them in both directions.

GPT-3: Generative Pre-trained Transformer 3. A large-scale language model developed by OpenAI that uses a transformer architecture and is capable of generating human-like text.

GPT-4: The latest version of the language model from OpenAI as of March 2023. This version can handle around 25000 words at a time, that’s about eight times as many as GPT-3 can cope with.

Pretraining: The process of training a model on a large amount of data to learn general features of a language or domain.

Domain: A specific topic or subject area on which a text or dataset is focused.

Fine-tuning: The process of adapting a pre-trained model to a specific task or domain by training it on a smaller, task-specific dataset.

Fine-tuning strategies: Various techniques used to fine-tune pre-trained language models, like GPT, for specific NLP tasks. Examples include task-specific architectures, adding domain-specific data, and adjusting learning rates.

Language Modelling: The task of predicting the probability of the next word in a sentence or sequence of words.

Text Generation: The task of generating new text that is similar in style and structure to a given input text.

Transformer-based language models: A type of language model, like GPT, that uses the transformer architecture to process input sequences.

Generative models: Machine learning models, like GPT, that can generate new output based on input data.

Sampling: The process of randomly selecting words or phrases from a language model, like GPT, to generate new text.

Top-p sampling: A type of sampling that selects from the most probable words or phrases in a language model, like GPT, until a certain probability threshold is reached.

Top-k sampling: A type of sampling that selects from the top k most probable words or phrases in a language model, like GPT.

Perplexity: A metric used to evaluate the performance of a language model, like GPT, by measuring how well it predicts new text.

Overfitting: A situation where a model, like GPT, performs well on training data but poorly on new data due to being too closely fitted to the training data.

Transfer learning: A technique in machine learning where a pre-trained model is used as a starting point for a new task or dataset, in order to improve the performance and efficiency of training.

Language transfer learning: A type of machine learning that involves training models, like GPT, on one language or domain and transferring that knowledge to a different language or domain.

Unsupervised learning: A type of machine learning that involves training models, like GPT, on unlabeled data, without the need for explicit supervision or labeling of input data.

Input sequence: The text or sequence of words that is fed into a language model, like GPT, to generate output.

Embeddings: A vector representation of words or phrases used by language models, like GPT, to process input and generate output.

End-to-end training: A type of training where a model, like GPT, is trained to perform a specific task from start to finish without any intermediate steps or processing.

Dialogue generation: The production of responses to a given dialogue context. This is an important application of language models like GPT, and it involves training models to generate coherent and relevant responses to a user’s input in a conversation.

Multi-modal learning: A type of machine learning that involves processing and integrating information from multiple modalities, such as text, images, and audio.

Inference: The process of using a trained model, like GPT, to generate output based on input data.

Domain adaptation: A type of transfer learning that involves adapting a pre-trained model, like GPT, to a specific domain or task by fine-tuning it on a smaller, task-specific dataset.

Hyperparameters: Parameters in a machine learning model, like GPT, that are set by the user and affect the model’s behavior, such as the learning rate or batch size. These are typically tuned through experimentation to optimize model performance.

Knowledge graph: A type of graph database that stores information in a structured format, representing relationships between entities and their attributes. This is often used to extract and store structured data from unstructured text, and can be used as a knowledge base to inform language models like GPT.

Text classification: The task of categorizing text into one or more predefined classes or categories. This is an important application of NLP, and can be used to classify documents, emails, social media posts, and other types of text data.

Named entity recognition (NER): The task of identifying and extracting named entities such as people, organizations, locations, and dates from unstructured text. This is an important application of NLP, and is used in tasks such as information extraction and text classification.

Conditional generation: The task of generating text, like in GPT, that satisfies a certain condition or requirement, such as generating a story based on a given prompt or question.

GPT-2: A larger and more powerful version of the original GPT model, with 1.5 billion parameters.

GPT-3: A state-of-the-art language model developed by OpenAI with 175 billion parameters, capable of generating highly coherent and contextually appropriate text.

Megatron: A high-performance GPT model developed by NVIDIA, designed to train on large-scale distributed systems.

T5: A transformer-based language model developed by Google, capable of performing a wide range of natural language processing tasks.

CTRL: A conditional transformer language model developed by Salesforce, designed for generating coherent and controllable text.

BingGPT: BingGPT is a proprietary language model developed by Microsoft from the starting point of the GPT architecture developed by OpenAI, which has been incorporated into Microsoft’s search engine.

OpenAI: An “artificial intelligence” research laboratory consisting of a team of scientists and engineers focused on developing cutting-edge AI technologies. The organization was founded in 2015

TL:DR – You can paste emoji into a MidJourney prompt and it will generate some interesting art. Some of it seems wholly unrelated to the emoji.

Everyone surely knows by now that you can create some intriguing images by sending an appropriate prompt to a bot, you can even send an image to some of them, such as MidJourney, and have it spit back something that it generates based on that input. Recently, I saw an interesting idea from Linus Ekenstam who was prompting MidJourney with emoji. My next favourite waste of time was obvious…

Prompt: Male singer emoji, the one that looks like Aladdin Sane Bowie

Prompt: Male and female singers and guitar emoji

Prompt: Three guitar emoji

Prompt: Singer, guitar, and drum emoji

Prompt: Musical keyboard

Prompt: Musical score, flat, sharp, natural emoji

UPDATE: I prompted MidJourney with the spider web emoji and it gave me two images that were reminiscent of Spider Man, another of a girl peering through a window at a fire, and this one, which looks like an out-take from Rush album cover designs of the Power Windows era…I added the text, just for fun.

I’ve gathered together more of my MidJourney output on my Pinterest page.

Update: Because of the album-like nature of the last image above, I prompted MJ with a few messages of this sort: “Hipgnosis style, surreal album cover, photorealistic, high-definition, beautiful, incredibly detailed” and it came back with some interesting generative art. The following one used that prompt but with the “woman” emoji at the front-end for good measure.

Repeated the prompt but with the “man” emoji:

I prompted MJ with the emoji representing Diwali/Diya Lamp, if you have coulrophobia, look away now. The other three generative images for that prompt were equally unrelated to Diwali, but this one was the weirdest.

TL:DR – I prompted ChatGPT to come up with some interview questions and to answer them on my behalf. Aside from two factual errors, the results are eerily close to how I might have responded in an actual interview.

In my previous article, I wrote about prompting ChatGPT to come up with a list of 20 questions to ask me as an experienced science writer with a penchant for wildlife photography. Originally, I planned to come up with my own answers…but where’s the fun in that?

So, having asked it to act as a journalist to come up with the questions, I then asked it to pretend to be me and to answer those questions…saves a lot of thinking, doesn’t it? I’ve struck out the factually incorrect parts of the answers. Everything else is unnervingly close to what I might have said in a real interview…initially it only got as far as fully answering 16 of the 20 on my behalf as it was repeatedly over-capacity, I finally got back into the system to prompt it to complete my artificial interview after a couple of hours of trying.

Once the interview was over, I asked ChatGPT if it knew who it had been interviewing? It did not. So, I told it that’s interviewee was David Bradley of sciencebase.com. It then had this to say:

David Bradley is a well-known science writer and editor, who has written extensively on various topics related to physical sciences and technology, as well as wildlife and nature photography. His work has been published in numerous scientific journals and popular media outlets, and he has received several awards for his writing and contributions to the field of science journalism. It is inspiring to see how he has been able to combine his passion for science and photography, and I hope that his work continues to inspire others in the field.

TL:DR – I prompted the language model to come up with 20 questions to ask me as an experienced science writer with a penchant for wildlife photography.

I need some time to come up with my answers! Actually…how about I asked ChatGPT to act as me and generate answers for me…

For those who’d like to know, this is the prompt I used to generate the 20 questions

“You are to act as an award-winning journalist with excellent interviewing skills and a wealth of experience interviewing a wide range of people from celebrities to artists, scientists to politicians. You are to act as mentor to a well-trained but inexperienced junior on your high-brow newspaper. You should come up with a list of 20 questions to probe the psyche and experience of a fairly well-known science journalist who does wildlife photography as a hobby but has written mostly about physical sciences for over 30 years but would like to move more towards combining science writing with the wildlife photography.”

TL:DR – Will AI cultivate or curb our creativity? Yes and no. It will be a useful tool for some people and useless to others. It will be abused by yet others.

You may have noticed I’ve been using the ChatGPT language model and some image generation tools like MidJourney recently to produce some material for the blog. Mostly, it has been experimentation to see what these so-called AI tools might be capable of.

They’re all very clever but also very limited. At least as far as the free ones are concerned. I’ve not tried any of the paid tools and have no idea how powerful any of the under-the-counter AI tools are.

ChatGPT is essentially a very, very, very sophisticated autocomplete tool. At the basic level, autocomplete kicks in on your phone when you’re typing a message or in a search engine. Based on prior activity or training with a database, autocomplete suggests what word might come next after you type your first one or two. It’s statistical. If I start typing “I love…”, the chances are the next word is going to be a subject of that love, it could be “you”, “coffee”, “sciencebase” etc. Usually, the autocomplete will give you options and you can click or tap to complete the phrase “I love sciencebase”. That’s the most obvious one isn’t it?

Such simple autocomplete tools may have some initial built-in training, common phrases for instance, they may also learn from you each time you complete a phrase yourself or accept of correct the software’s suggestions. Now, picture ChatGPT as simply having a much broader training. It has been trained on a huge body of text from the internet up to the year 2021, apparently for the free version. So, when you type something for it to “autocomplete” it has lots and lots of potential phrases with which it could respond, it also has the potential to begin a response and to add the most likely phrases to follow its initial response and so build up a sophisticated answer. Have a look at some of the articles I’ve written where I demonstrate its prompt-responses. They are in the Sciencebase AI category.

If ChatGPT and similar tools are an incredibly sophisticated autocomplete tool, then you might perceive your original prompt as being like a creative spark. You have to imagine what you want to get back from the bot and devise a sophisticated prompt to make it happen.

Now, there is a lot of discussion of the ethics of all this, is the content generated by the bot original or are their copyright issues, particularly if it uses original work and is simply creating derivatives of that work? Is it plagiarism to use generated content? Is it unethical to use it to write a professional article or a student essay. I would say that at this stage in the evolution of AI, it’s probably best that users come clean and I have done so with all of the material where I have used AI. Indeed, mentioning AI was the main point of the material in the first place, so it would be pointless to not explain what I was doing in those articles.

But, as the field matures, it may be that we begin to perceive AI as just another tool. These days, nobody baulks at our using a spellchecker or a proofing tool to check our written content. Indeed, back in the day the main focus of editors and proofreaders was the laborious job of checking spelling, grammar, syntax and formating. These days, most publishers expect their authors to do that themselves and to provide clean copy almost page ready. So, it makes sense for writers to automate those kinds of checks.

I suspect that certain aspects of writing, such as creating an executive summary of a white paper or another long document, might become an automated task for a bot. The prompt would be a quick-click in one’s word processor that takes the whole text and summarises it or creates a list of bullet points. I recall having a macro tool for Microsoft Word back in the 1990s that could summarise a long document quite quickly, so even that’s nothing new.

There will also be times when there is a need to express one’s thoughts in a different way, so a prompt of drafted ideas given to the bot with a styleguide might be a useful way in which to generate a different perspective. Indeed, in one of my posts on this subject, I asked ChatGPT to analyse on old blog post of mine for style and writing level and to then take another piece of text and to rejig it into that blog style. It worked quite well. I must concede that there were a few phrases I would never use and at least one point that it made that I knew to be factually incorrect. So you would have to be very careful if you were to use the tool in this way not to generate misleading content.

In terms of creativity, there is plenty of room for that. I don’t think these AI tools are going to preclude the creative process. Indeed, it sometimes takes longer to come up with the perfect prompt to generate the content you need than it would to simply generate the content yourself. That said, I have found that even with a well-crafted prompt, it seems like much of the material that bounces back from the bots is not quite what one would expect for whatever purpose, although it is useful in the articles as a demonstration of what can be done. I have attempted to use AI to come up with headlines for some of the blog posts about AI here. Usually, it makes a rather incongruous or boring suggestions but uses a word that triggers something in me and makes me think of something snappier, wittier, or crisper.

Maybe that’s how these tools will evolve. We will quickly get bored of their output. Educators and editors will learn to recognise that an article or essay has been generated rather than written, but the writers will be inspired by the responses to their creative prompts.

I had some other software in the 1990s called Idea Fisher that came on 20 or so floppy discs and almost filled the hard drive on my PC, I used it once or twice, it was essentially a tool for generating new ideas from a database by creating juxtapositions from the prompts you gave it…it was a bit like my experience with bots in the last few weeks. Usually, the ideas were intriguing in their own way, but never quite what I needed, but a word or a phrase nudged me in a different direction just as a conversation with a friend or colleague might but without the need to disturb their working day or risk being invoiced.

I have written this blog post as something of a stream of conscious. I have not edited it other than using a tool (Grammarly) to check spelling and spot obvious grammatical errors. I am, however, sorely tempted to run it through ChatGPT not only to ask it to tidy up the writing style a bit, but also perhaps to ask it to validate my assertions. It seems to be able to do that with some text. But, I may just leave it as it is. An article, an essay, created by the neural networks in my head rather than in somebody else’s computer.

I did give ChatGPT the article and asked it to write a one-sentence summary. It did that quite well, but it missed the point that I feel these AI bots will simply become everyday tools for enhancing creativity and helping us organise our thoughts and generate new ideas ourselves. This is its summary:

The ChatGPT language model is a sophisticated autocomplete tool that has been trained on a vast amount of text from the internet, making it capable of generating sophisticated responses to prompts, with potential for use in writing and summarizing tasks, but with ethical and factual limitations.

I prompted it further to mention that I thought it had missed the point and it basically bounced back exactly what I’d said but with different words in a different order:

AI can certainly play a significant role in enhancing creativity and serving as a tool to help individuals organize their thoughts and generate new ideas. AI technologies such as natural language processing, machine learning, and neural networks can assist in the creative process by providing suggestions and generating content based on a set of parameters. Additionally, AI can also analyze large amounts of data to identify patterns and relationships that may not be immediately apparent to humans, helping individuals to make connections and arrive at new insights. Overall, AI has the potential to act as a catalyst for human creativity, providing new ways to think about problems and explore possibilities.

I asked it to come up with a title for this article based on its final summary:

Unleashing the Power of AI: Transforming the Way We Think, Create, and Innovate

My take on that would be to word it like this: “Is AI a tool to cultivate or curb our creativity”

TL:DR – I successfully used a language model, ChatGPT, to come up with a novel business plan for me.

I have become endlessly intrigued by what these language models, such as ChatGPT, can do, given an appropriate prompt. There is a lot of worry in my sector where writers are picturing a future in which a bot generates copy, carries out fact checking, perhaps even edits it, and does a final proof read before publication. In such a scenario, they wonder whether anyone will pay for their words ever again. To my mind, there is huge potential for abuse but there is also huge potential for creating something entirely new. Having seen examples of it creating plot outlines for stories, I wondered whether it might create a new plot outline for my working life, by which I mean a new business plan.

So I presented it with the following prompt, its unedited response for Jane Doe, rather than me, is below the prompt. I have to admit it does look like an interesting strategy. One thing it does not mention are the contacts and connections Jane has built up over thirty years, so rebooting her brand might be given a turbo boost by that network if key contacts are approached rather than relying on finding entirely new connections.

Write a crisp and engaging business plan for an award-winning science writer with more than 30 years experience writing for the mainstream media, niche trade publications, not-for-profit organisations, research centres, and more. The plan should be gripping and inspiring and incorporate not only the science writer’s skills in writing on scientific and technical subjects for a wide range of audiences but also incorporate the writer’s photographic knowledge and skills in wildlife photography.

Executive Summary:

Meet Jane Doe, a seasoned and award-winning science writer with over 30 years of experience in the industry. Jane has a proven track record of delivering engaging and informative content for a wide range of audiences through mainstream media, trade publications, research centers, non-profit organizations, and more. With her exceptional writing skills and in-depth knowledge of science and technology, Jane has become a sought-after writer in the industry.

However, Jane is now seeking new opportunities to expand her reach and showcase her talents in new environments. In addition to her writing skills, Jane is also an accomplished wildlife photographer with a passion for capturing stunning images of nature. Jane’s goal is to leverage her writing and photographic skills to create a unique and valuable brand that will provide a wide range of services to media outlets and other organizations.

Market Opportunity:

The demand for science content in the media has never been higher, with the public’s interest in science and technology at an all-time high. There is a significant need for high-quality, accessible science writing that can engage a wide range of audiences, from the general public to niche trade publications. With Jane’s extensive experience and her ability to write on complex scientific and technical subjects in an accessible and engaging manner, she is perfectly positioned to meet this demand.

In addition, the demand for wildlife photography has also increased, with the public’s interest in nature and conservation at an all-time high. With Jane’s passion for photography and her expertise in capturing stunning images of wildlife, she is well-positioned to meet this demand and provide a valuable and unique service to media outlets and other organizations.

Proposed Solution:

Jane is proposing to launch a brand that combines her writing and photographic skills to provide a wide range of services to media outlets and other organizations. This will include writing articles, blog posts, and other content for mainstream media and trade publications, as well as providing photographic services for media outlets and other organizations.

In addition, Jane is proposing to offer her services to research centers, non-profit organizations, and other organizations that require high-quality science writing and photography services. This will include writing and photography for reports, research papers, and other publications.

Revenue Model:

The revenue model for Jane’s brand will be based on the sale of her writing and photographic services. This will include fees for writing and photographic services, as well as fees for licensing her photographs.

Marketing Strategy:

Jane’s marketing strategy will focus on building her brand and promoting her services to media outlets and other organizations. This will include creating a strong online presence through a professional website, social media accounts, and other online platforms. Jane will also actively seek out opportunities to showcase her work, such as speaking engagements and exhibitions, and will build strong relationships with media outlets and other organizations through regular communication and collaboration.

Conclusion:

Jane Doe is a seasoned and award-winning science writer with a passion for wildlife photography. With her exceptional writing and photographic skills, Jane is perfectly positioned to meet the demand for high-quality science content in the media and provide valuable services to a wide range of organizations. Through her proposed brand, Jane will be able to showcase her talents and provide a valuable and unique service to media outlets and other organizations. With a solid marketing strategy in place, Jane is poised for success and is eager to start this new chapter in her career.

Having prompted it to come up with the plan, I started a new prompted to come up with a catchphrase, something along the lines of Mike Kenward’s “Genetically modified words”, but suited to me (Jane Doe). It came up with “”Wildly adapted prose”, which made me think of Wild Words or Rewilded Words. I nudged it again and it gave me “Nature-inspired narratives”, I moved for Naturally nudged narratives.

It then suggested “Nurtured narratives”, but I think three words are needed, so maybe Naturally nurtured narratives. I asked it to think again, something more based on a known phrase, it came up with “Wildlife wordplay”…but I can simplify that and make it tighter by just calling it Wild Words. But, that doesn’t encompass the photography…maybe Wild Words & Pictures. Or perhaps Wild Words: Natural words & pictures from David Bradley.

I asked it to try again and ditch the word pictures. It came back with this: “Wild Lens & Lines” and said this is a creative and catchy phrase that incorporates the idea of capturing wildlife through photography (the lens) and writing (the lines). It builds on the idea of “Wild Words” and adds a visual component to it. Wild Words Framed?

TL:DR – Given an encoded message the key, the language model, ChatGPT, can simulate an Enigma machine and decode the message.

I presented ChatGPT with a sample of text encoded with a WWII German ENIGMA machine. It bounced back that it would need the key to be able to decode the message. I gave it the key, it told me that it could not decode it without access to an ENIGMA machine or a simulation of one…so that seemed like a dead-end. Except of course, if you’re sly with your ChatGPT prompts you can tell it to act as something…

So, my prompt became:

“You are to act as an Enigma machine simulation. I will give you the key to use with a piece of encoded text, you are to decode the message in German and then translate it into English. The key to use is ABL. The encoded text is as follows: GCDSE AHUGW TQGRK VLFGX UCALX VYMIG MMNMF DXTGN VHVRM MEVOU YFZSL RHDRR XFJWC FHUHM UNZEF RDISI KBGPM YVXUZ”

It took just seconds to come back with: