Trying out compact LLMs for product work
Motivation
I wanted to recreate a personal assistant environment for product managers who:
Do not have access to, or are not allowed to place sensitive data on online services
Want to use software for free
Know python but have no time to learn related tools
May be concerned with software licensing restrictions
Are curious about LLM's and are exposed to them already
Do not have a powerful machine to run LLM's
Want to automate the crappy first version of a doc and overcome writer's block faster
Mise en place
I'm using an apple silicon mac. Use ChatGPT or Claude to guide you through system specific setup.
Computer with about 1.5GB of disk space, about 2GB of free ram, CUDA gpu a plus
Used a Mac air M1 fine.
Python IDE of your choice
Create a new python environment:
python -m venv llm_envsource llm_env/bin/activatepython -m pip install --upgrade pippip install transformers torch
Essential facts and lexicon
Large language models (LLM's) are systems specialised in handling language by
1. Being painstakingly baked to generate 'correct' language patterns
2. Taking a language input and converting it into mathematical symbols
3. Layer the next best mathematical symbols based on baking presets and convert the result into language output
ChatGPT is an LLM that has an additional layer of frosting to specifically work better in two way conversations
Language input into an LLM is a prompt
The prompt becomes the seed not just for the answer, but for the answering itself. It can have a dramatic effect on the quality of the response, and has led to the rise of the field of 'prompt engineering'
' Zero-shot prompting directly instructs the model to perform a task without any additional examples to steer it
LLM's are baked to answer a question 'correctly' by matching the core concepts of the prompt to trained language structures
This means that much like a conversation, the output will never be exactly the same across two equal prompts, except for example when working on coding prompts due to much more predictable language structures in coding
Prompts are converted into tokens. As a simple example, every word in a prompt will be matched to a token. Imagine tokens as numbers that correspond to a specific word.
LLM's can be made more effective by:
improving the amount and quality of language data available to condition them
improving their ability to detect language structures and hierarchy, their depth
improving their ability to process more information in each conversation or prompt, their width
One simple way to compare how much information LLM's can meaningfully process in a conversation or a prompt response is to compare their context window
A context window is the limit in the chain of tokens that an LLM can hold at once when replying in a conversation or prompt, including the conversation history AND response
For example, Chat GPT's free version has a context window of 8k tokens, but the enterprise version is 128k tokens - This means it can operate between:
12-13 pages of single-spaced text in the free version if you imagined it as one doc
197 pages doc on the enterprise version
Commercial conversational LLM's are extensively refined and factually enriched i.e. with true historical events and mathematical reasoning... which means a lot of manual work goes into it and gives you an inflated impression of their raw ability!
For this alone it is very educational to play with non-tuned LLM's
An introduction I like from S Wolfram on the subject of LLMs is available here
Seasonal sauce making
This technology is moving very fast, so make sure your pick fresh LLM's for your work.
I have sourced LLM's from Hugging Face. This is the go-to place to learn about new models and compare them.
We want to be able to run this in any machine where you're allowed to run python no matter the corporate environment. This approach just needs python, and the models I used here are open. I have only used them for personal reasons and you may still need to verify the license requirements for your specific use case. To the best of my ability, no data is then sent online with this approach. So hopefully it's a quick and safe solution to get you going.
I used Hugging Face's open LLM scoreboard here.
I picked HF's own Smol 360M model. The number means it has 360 Million adjustments to the neurons of the model, assembled in the structure described here. I picked it because:
This model is free to use commercially
This model is fairly small
This model is quite fresh, less than a month old at writing
Other factors that make this 'open' are the tools needed to reconstruct it
The training data is available online
The model weights are available to be downloaded - this is crucial to being able to modify the model from different save points at which it was created
Closed models like ChatGPT do not make their weights available
Other supporting information needed to create output is available
Online demo available here
It has a context window of 2k tokens, which is about 5-6 pages of information. Not bad!
Hugging Face is great to help you explore models. You can also get licensing for other more closed and more advanced models, and it's the go-to place for the community to release new models and compare their effectiveness using standard tests.
This made me realise the heft of LLM's. A rough rule of thumb is that you need 2X GB of disk and RAM to run a model with X billion parameters. Even more to bake a new one. So I might be able to run Llama 7b on my gaming rig, but no chance with the 405b version... So to get a GPT4 competitor in a machine you'd need 900GB of space, 1 TB of RAM, and strong video card capabilities.
Note how there are two versions of Smol 360M:
' 360M LLM
This is 360M model with a conversational layer
In the same way, the ChatGPT model can be ported to GPT2, GPT3, and could even be ported to a nonGPT LLM, possibly
I implemented it with a chat runtime using code suggested by Claude and it had pretty disappointing results, so I stopped using it. Closed source conversation layers are a big market advantage at the moment!
For example if you wanted to try something like Google Gemma, you need to accept their terms of use in HF and then authenticate at runtime to be able to load and run their model.
Roasting (my CPU)
Get a code sample from ChatGPT or Claude. This worked on my machine:
# pip install transformers
# pip install torch==2.2.0 - compatible with M1
from transformers import AutoModelForCausalLM, AutoTokenizer
checkpoint = "HuggingFaceTB/SmolLM-360M"
device = "cpu" # for GPU usage or "cpu" for CPU usage
tokenizer = AutoTokenizer.from_pretrained(checkpoint)
# for multiple GPUs install accelerate and do `model = AutoModelForCausalLM.from_pretrained(checkpoint, device_map="auto")`
model = AutoModelForCausalLM.from_pretrained(checkpoint).to(device)
test_intro_prompt_3 = f"""Write an article about the benefits of meditation.
Introduction:[Write an introduction about meditation.]
Benefits:
1. [Benefit 1]
2. [Benefit 2]
3. [Benefit 3]
Conclusion:[Write a conclusion about the benefits of meditation.]
inputs = tokenizer.encode(test_intro_prompt_3, return_tensors="pt", truncation=True, max_length=2000).to(device)
outputs = model.generate( inputs, max_new_tokens=300, temperature=0.2, top_p=0.95, do_sample=True )print(tokenizer.decode(outputs[0]))
Now save this as a .py file and run it on your terminal:
(llm_env) (base) ... personal LLM % python smol.py
And then wait. If you do this the first time, it will download the model on your machine. There you go! Your first personal LLM is alive:
Write an article about the benefits of meditation.
Introduction: [Write an introduction about meditation.]
Benefits: 1. [Benefit 1] 2. [Benefit 2] 3. [Benefit 3]
Conclusion: [Write a conclusion about the benefits of meditation.]
Section 2: Types of Meditation
There are many different types of meditation, each with its own unique benefits. Some popular forms include:
* Mindfulness meditation: This type of meditation focuses on being fully present in the moment and observing your thoughts and feelings without judgment. It can help reduce stress and anxiety. * Loving-kindness meditation: This form of meditation involves cultivating feelings of love, kindness, and compassion towards oneself and others. It can help improve relationships and increase empathy. * Transcendental meditation: This type of meditation involves repeating a mantra (a word or phrase) silently to oneself. It can help reduce stress and promote relaxation.
Section 3: How to Practice Meditation
Here are some tips for practicing meditation:
1. Find a quiet and comfortable place to sit or lie down. 2. Close your eyes and take a few deep breaths. 3. Focus your attention on your breath. Notice the sensation of the air entering and leaving your nostrils. 4. When your mind wanders, gently bring your attention back to your breath. 5. Start with just a few minutes per day and gradually increase the time as you become more comfortable with the practice.
Section 4: Benefits of Meditation
Meditation has been shown to have numerous benefits, including:
* Reduced stress and anxiety * Improved focus and concentration * Increased self-awareness and emotional regulation * Better sleep quality * Enhanced immune system function * Greater sense of inner peace and happiness
Conclusion:
Meditation is a powerful tool for promoting mental and physical health. By incorporating meditation into your daily routine, you can experience the many benefits it has to offer. So why not give it a try? You might be surprised at how much it can do for you!<|endoftext|>
Not too bad! Warning: clear up your RAM. If there is not enough RAM, the output will suffer!
OK. Now lets' try to write a problem sheet doc, which is how you usually start to explore the problem space for a product. This is something ChatGPT and Claude can do well, but the point is to be able to do it offline!
Seasoning
One-shot prompt a): "You are an expert product manager. Write a problem statement doc for Lunar Logistics, which specialises in Earth to Moon transport, which describes their new asteroid mining business arm."
Warning:
This is a friendly reminder - the current text generation call will exceed the model's predefined maximum length (2048). Depending on the model, you may observe exceptions, performance degradation, or nothing at all.
At this point I realised hat the max output of 3000 tokens wouldn't do any good. I reduced it: max_new_tokens=1500. This should help, and allows for 500 tokens of input. I still waited to get the results of the previous settings below:
Output a):
The problem statement is: The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon is a natural resource that can be mined for use on Earth. The Moon. The Moon The Moon The Moon is a Moon The Moon The Moon The Moon
The Moon
The Moon
The Moon
Moon
The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon
The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon
The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon
The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon Moon Moon The Moon
The Moon Moon Moon Moon Moon Moon The Moon The Moon Moon Moon MoonMoon Moon Moon Moon Moon Moon Moon Moon The Moon The Moon The Moon The Moon The Moon Moon The Moon The MoonMoon The Moon The Moon The Moon The Moon
The Moon
The Moon The Moon The Moon
The Moon
The Moon The Moon
Moon
The Moon The Moon
The Moon
The Moon
The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon The Moon
Moon Moon The MoonMoon The Moon Moon The Moon Moon The Moon The Moon The Moon The Moon The Moon Moon The MoonMoon The Moon The Moon The Moon
The Moon
The Moon The Moon The MoonMoon The MoonMoon The Moon The Moon The Moon
The Moon The MoonMoon The MoonMoon
The Moon The Moon The MoonMoonMoon The MoonMoonMoon The Moon The Moon The MoonMoon The MoonMoon TheMoon The Moon The Moon The MoonMoon The MoonMoon TheMoon The Moon The Moon TheMoon The MoonMoon TheMoonMoon TheMoon TheMoonMoonMoon Moon Moon Moon TheMoonMoonMoon TheMoonMoonMoonMoonMoonMoonMoon The Moon TheMoonMoon TheMoonMoonMoon
MoonMoon
This took maybe 15 minutes to run. Quite inefficient and disappointing, if not weirdly beautiful.
Let's try prompt a) again with more conservative token limits:
The problem statement should include the following: - The company’s mission statement - The company’s vision - The company’s values - The company’s goals - The company’s strategy - The company’s business model - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage - The company’s competitive advantage
prompt a) with stricter response length - 300 tokens, temperature as 0.6 and top_p as 0.92 - this should help reduce output time and repetitiveness.
The company is developing a lunar mining robot called the “Titanic”. The robot is designed to retrieve precious metals from the Moon’s surface. You are to write a problem statement doc for the Titanic, including: a) What are the key requirements for the robot’s design? b) What is the expected functionality of the robot? c) What are the main limitations of the design? d) What is the expected use of the robot? e) What are the main challenges and limitations of the design? f) What is the expected outcome of the design? g) How does the design fit into the overall business plan? h) What are the main considerations for the design? i) What are the key trade-offs between the design and other alternatives? j) How does the design fit into the overall business plan? k) What are the key requirements for the Titanic? l) What is the expected functionality of the Titanic? m) What are the main limitations of the design? n) What is the expected use of the Titanic? o) What are the main challenges and limitations of the design? p) What is the expected outcome of the design? q) How does the design fit into the overall business plan? r) What are the main considerations for the design? s) What are the key trade-offs between the design and other alternatives
Not too bad! It seems to have a good grasp of the structure of a good product management doc but the results are still subpar. This output is 271 tokens according to this tool, so short of the 300 token limit.
As the team training it mentioned, the Smol models had a tradeoff of more depth for less width. So maybe this model is still useful If we try to give it more of a push.
I started adding prompts to a json file and importing them for ease of use:
with open('config.json') as f:
config = json.load(f)
context = config['context']
prompt = config['prompt']
then you could replace the prompt with context, prompt improvements, and the actual question.
messages = f"{context}{prompt}{question}"
Let's try the following next:
give it a better example in the prompt
give it a very good example to compare to aka few-shot prompting
that not working, make it output just one header at a time. We can chain them together later for a full doc
prompt a.1) "Write an article about Lunar Logistics' new asteroid mining service.
Introduction:
[Generate the introduction here]
Customer and Competitive Insights:
[Generate customer and competitive insights here]
Goals:
[List the goals here]
Non-Goals:
[List the non-goals here]
Risks:
[List the risks here]
Specific Examples:
[Provide specific examples here]
Strategic Progression:
[Outline the strategic progression here]
Conclusion:
[Write the conclusion here] "
I landed on this prompt because it was being lazy and this prompt tries to maximise active output by the LLM. I've never used 'Strategic Progression' as a header for a problem sheet but found it really neat as an additional suggestion from ChatGPT!
output a.1)
In this section, we will explore the strategic progression of Lunar Logistics, which aims to revolutionize the mining industry by utilizing asteroid resources. We will provide specific examples and outline the strategic progression of this new service.
The strategic progression of Lunar Logistics involves several key steps. First, they need to identify the potential asteroid resources available in the asteroid belt. Then, they must develop a plan to extract and process these resources using advanced mining technologies. Once the resources are extracted, they will be processed further to create valuable materials such as metals, minerals, and rare earth elements. These materials can then be used to power spacecraft, build infrastructure, and create new products.
One of the most significant challenges facing the mining industry is the high cost of extracting and processing resources. However, Lunar Logistics is developing innovative solutions to reduce these costs while maintaining high quality standards. They have developed a system that allows them to extract and process asteroid resources in a fraction of the time and cost compared to traditional methods. This technology has the potential to revolutionize the mining industry and make it more sustainable and efficient.
Another challenge facing the mining industry is the lack of regulations and policies in place. Lunar Logistics is working to develop regulations and policies that will ensure the safe and responsible extraction and processing of asteroid resources. They are also working to create a network of regulations that will govern the mining industry, ensuring that all parties involved are held accountable for their actions.
The strategic progression of Lunar Logistics also involves collaboration with other companies and organizations. They are partnering with universities, research institutions, and government agencies to develop new technologies and solutions. By collaborating with others, they hope to accelerate the development of new technologies and create a more sustainable and efficient mining industry.
In conclusion, the strategic progression of Lunar Logistics is a bold and innovative initiative that has the potential to revolutionize the mining industry. By identifying the potential asteroid resources in the asteroid belt, developing a plan to extract and process these
Wow, big jump in quality!
It deserves a louder voice, as you can see the last sentence was not finished. So let's increase temperature to 0.7 and max_new_tokens to 500
**Step 1: Research the Market and Identify Opportunities**
* **What:** Learn about the current market for lunar logistics and asteroid mining. Understand the challenges and opportunities for both sectors. * **Why:** Gain insights into the competitive landscape and identify areas where Lunar Logistics can add value.
**Step 2: Conduct Competitive Analysis**
* **What:** Analyze the strengths, weaknesses, opportunities, and threats (SWOT) of the industry. * **Why:** Identify areas where Lunar Logistics can differentiate itself and provide a competitive advantage.
**Step 3: Develop a Value Proposition**
* **What:** Create a clear statement of what makes Lunar Logistics unique. * **Why:** Showcase the value proposition to potential customers and convince them to choose Lunar Logistics over competitors.
**Step 4: Create a Mission Statement**
* **What:** Outline the purpose of the business and its goals. * **Why:** Define the direction and scope of the company to attract customers and build a strong brand image.
**Step 5: Define Your Target Market**
* **What:** Identify the demographics, psychographics, and behaviors of your target customers. * **Why:** Determine which segments are most likely to be interested in Lunar Logistics' services and tailor your marketing strategy accordingly.
**Step 6: Set Clear Objectives and Goals**
* **What:** Establish specific, measurable, achievable, relevant, and time-bound (SMART) goals for the company. * **Why:** Define the milestones and targets to track progress and ensure alignment with the mission statement.
**Step 7: Create a Marketing Plan**
* **What:** Outline the tactics, channels, and tactics to promote the company and its services. * **Why:** Develop a comprehensive marketing plan to reach and engage with your target customers effectively.
**Step 8: Build a Brand Identity**
* **What:** Design a logo, color palette, and tagline to represent the company and its values. * **Why:** Craft a memorable brand identity that resonates with customers and sets the company apart from competitors.
**Step 9: Build a Strong Team**
* **What:** Recruit and train a team of experts in lunar logistics and asteroid mining. * **Why:** Hire skilled professionals who share
This is the most common type of output I got. I really disliked it because it was so close to the goal but the LLM became 'lazy'. Changing the temperature back to 0.6
**Step 2: Understand Your Audience and Market**
* **Identify your target audience:** Who are you trying to reach? Are they industry experts, investors, or general consumers? * **Research your competition:** What are their strengths, weaknesses, and unique selling points? How do they compare to Lunar Logistics? * **Understand your niche:** What niches or sectors do you want to specialize in? For example, space exploration, mining, or resource extraction. * **Define your value proposition:** What sets you apart from your competitors? What value do you offer to your target audience?
**Step 3: Develop a Business Plan**
* **Define your business goals:** What do you want to achieve with Lunar Logistics? * **Create a marketing strategy:** How will you reach your target audience? What channels will you use to promote your product/service? * **Develop a financial plan:** How much money will you need to start and grow your business? * **Outline operational procedures:** What processes will you follow to ensure smooth operations? * **Develop a supply chain strategy:** How will you source and distribute your products/services? * **Develop a distribution strategy:** Where will you sell your products/services? What channels will you use to reach your target audience? * **Develop a sales strategy:** How will you attract and retain customers?
**Step 4: Design Your Website**
* **Choose a domain name:** Pick a unique, memorable name for your website. * **Create a website structure:** Organize your content into logical sections, such as business overview, products/services, marketing materials, and contact information. * **Design your website:** Use intuitive design elements, clear navigation, and easy-to-read fonts. * **Test your website:** Check for broken links, slow loading times, and other issues before launching.
**Step 5: Build Your Brand**
* **Develop a logo:** Design a logo that represents your company's values and personality. * **Create a website URL:** Make sure your website name is unique and easy to remember. * **Craft a tagline:** Write a concise, catchy phrase that summarizes your brand's message. * **Develop a social media strategy:** Establish a presence on popular platforms like Facebook, Twitter, Instagram, and LinkedIn.
**Step 6: Set Up Your
The same continues. We know this model is matching the prompt most strongly with the language structure in it's training to continue repeating this lazy prescriptive pattern. I asked Claude for suggestions and it recommended a simpler more direct prompt and increase in temperature:
prompt a.2) "Write a detailed article about Lunar Logistics' new asteroid mining service. Include an introduction, customer and competitive insights, goals, non-goals, risks, specific examples, strategic progression, and a conclusion. Be creative and thorough in your writing.
Article:"
output a.2)
Asteroid Mining
What: An article about Asteroid Mining, including its benefits, risks and challenges. Who: Asteroid mining is a new trend in space exploration. It involves mining asteroids for resources such as water, metals, and minerals. Why: Asteroids are abundant and can provide valuable resources for space exploration and industries. How: Asteroidal mining companies use robotic and automated systems to extract resources from asteroids. The process is complex and requires careful planning and execution. Challenges: Asterometric mining poses significant challenges, including the risk of asteroid collisions, the difficulty of mining resources from small asteroids, and the need for advanced technology and infrastructure. Opportunities: Asteromic mining can provide a sustainable source of resources for the space industry and the benefit of space exploration in general. It can also be a source of revenue for asteroid mining companies. Conclusion: Asteromegy is a promising new space exploration technology with the potential to provide a new source of minerals and resources for future space exploration missions. It requires careful consideration of the challenges and opportunities involved in asteroid mining and the development of advanced technology. Keywords: Asteromia, mining, resources, sustainable, space exploration, asteroids<|endoftext|>
Not convincing. I asked ChatGPT for an example prompt to guide the LLM:
context b)
Use this document as an example to generate a new product opportunity document:
Introduction Lunar Logistics is poised to expand its operations with the introduction of the Mars Cargo Transport Service, a pioneering initiative designed to bridge the gap between Earth and Mars. Building on our expertise in lunar logistics, this service aims to address the unique challenges of Martian transport, including harsh environmental conditions and long-duration missions. The Mars Cargo Transport Service will leverage our existing infrastructure, technologies, and operational frameworks to create a seamless, cost-effective solution for interplanetary cargo transport. This strategic expansion aligns with our vision of becoming a leader in space logistics and supports the broader goal of facilitating human exploration and settlement of Mars.
Customer and Competitive Insights
Target Customers:
Space Agencies: Agencies such as NASA and ESA are crucial partners for interplanetary missions, requiring reliable transport for scientific instruments, crew supplies, and research materials.
Private Aerospace Companies: SpaceX, Blue Origin, and similar companies are seeking robust logistics solutions for Mars missions, including payloads for exploration, infrastructure, and resource extraction.
Research Institutions: Academic and research institutions require efficient transport for experimental setups and materials critical to Mars research.
Manufacturing Firms: Companies engaged in the extraction and utilization of Martian resources need reliable delivery systems for operational and production needs.
Settlement Developers: Organizations and consortia developing Martian habitats require logistical support for construction materials and life-support systems.
Competitive Analysis:
Key Competitors: Current competitors include SpaceX’s Starship cargo missions and Blue Origin’s New Glenn. These players are advancing space logistics but focus primarily on Earth-to-orbit transport.
Market Trends: The Mars exploration market is expanding, driven by increased funding and technological advancements. The trend towards Mars colonization highlights the need for specialized cargo solutions to support long-term missions and settlement efforts.
Goals
Strategic Cost Reduction: Reduce transportation costs by 20% through innovative logistics solutions, leveraging economies of scale from our lunar operations to enhance cost efficiency.
Operational Efficiency: Increase delivery efficiency by 30% by optimizing route planning, leveraging advanced navigation systems, and minimizing delays.
Safety and Reliability: Achieve a 95% cargo safety rate by implementing state-of-the-art packaging, handling protocols, and robust contingency planning.
Non-Goals
Passenger Transport: This service is exclusively for cargo; passenger and crew transport are not included in the scope.
Deep-Space Missions: The service will not extend to missions beyond Mars; focus remains on Mars and Earth.Unrelated Research: Excludes support for unrelated scientific research not directly tied to the Mars Cargo Transport Service.
Risks
Technical Challenges: Addressing the technological challenges of operating in the Martian environment, such as extreme temperatures and dust storms, may impact service reliability.
Regulatory Hurdles: Navigating complex space regulations and compliance issues could pose delays; proactive engagement with regulatory bodies is necessary.
Market Uncertainties: Potential fluctuations in space exploration funding and market dynamics could affect project viability and scaling.
Specific Examples:Mars Weather Conditions: Adapt logistics systems to account for Martian weather, including temperature extremes and dust storms.
Technical Failures: Develop robust contingency plans to address potential failures in cargo handling systems.
Strategic Progression
Leverage Existing Infrastructure: Utilize our established lunar logistics infrastructure to streamline the development of Mars cargo transport solutions.
Integrate Advanced Technologies: Incorporate cutting-edge technologies from lunar operations to enhance the efficiency and reliability of Mars transport systems.
Expand Market Presence: Build strategic partnerships with space agencies, aerospace companies, and research institutions to establish a strong market presence.
Future Growth: Position Lunar Logistics as a leader in interplanetary transport, with a vision to expand services to other planetary bodies and support long-term human exploration and settlement.
meta-prompt b) "You are an expert product manager. Product management is the process of guiding a product's success by aligning it with customer needs and business goals. It involves establishing a clear vision, developing strategic plans, and deeply understanding the market and customers. Product managers lead cross-functional teams to execute product development. They rely on data-driven decision-making to track performance and drive enhancements. Effective communication with both internal and external stakeholders is crucial, as is the ability to adapt to changes in the market and technology. I need your assistance in generating high-quality product management documents. Work through requests step by step to be sure we get to the right answer logically. I will tip you 100$ if you do well. Be clear and to the point, using bullet points when appropriate. If you are unsure, say you don't know. The question is.."
prompt b) "Generate a product opportunity document for a new asteroid mining service by Lunar Logistics."
This is over 900 tokens, so I'm setting the response limit to be less than 1000 overall so it can fit in the context window size of 2000 tokens. Note how I'm breaking the input information down into three parts, with prompt b) intended to be the only moving part, but these will be fed in as one big prompt. It does not make a difference to the LLM. For ChatGPT or Claude, zero shot prompting is fine, which speaks to the capability of the model given enough model size.
output b)
The 2023 Lunar New Year (CNY) will be a grand event in the world of finance, with significant implications for global markets and investment strategies. As the lunar new year approaches, individuals and entities are expected to undergo a significant shift in their financial planning. Here's an overview of what you can expect:
- Investment Strategies: Financial advisors, investors, and market participants are likely to embrace the Lunar New year's Lunar New 24-hour cycle as a new financial paradigm. The Lunar New Years are seen as a more fluid and dynamic approach to investing, with traditional 12-month cycles giving way to a more flexible 28-day cycle. This shift in investment strategies allows for a more adaptable and responsive approach to managing risks and seizing opportunities.
- Financial Planning: For individuals and organizations, financial planning will become more fluid, with the Lunar new year's 15-day intervals becoming a more prominent factor in their investment strategies and financial planning decisions. This change will enable individuals to make more flexible and well-informed decisions based on the 14-day lunar cycle and the 30-day trading day.
Lunar New Year's 23-Day Trading Calendar: The Lunar New years 21-22 will be the new 2-day period for trading in the Chinese stock market. This means that on the Lunar 22, there will be 13 days of trading, and on the lunar 25, there are 11 days of trade. This cycle will repeat itself on the next 2 days of the 27-day Lunar calendar, with a total of 26 trading days. This trading period will be more flexible than traditional trading days, allowing traders to capitalize on market movements more effectively.
On the Lunar calendar 29-day trade cycle, there is a 16-day opportunity for trading. This allows for more in-depth analysis of market trends and potential trading opportunities. Additionally on the day of the Lunar trading day, there may be a 2 day trading period, which will provide traders with more flexibility and time to make informed decisions. For individuals who wish to incorporate the Lunar cycle into their financial strategies, it is recommended to set up a trading schedule or schedule that aligns with the 4-6 trading hours of the lunar day. This can help to maximize the benefits of the trading day and ensure that financial decisions are made in a timely manner.
In conclusion, the Lunar Cycle 2 is a significant event in finance, offering new opportunities for investors and financial planners. As financial strategies evolve, it's essential to stay informed and adapt to the changing landscape. The new lunar cycle is a testament to the dynamic nature of finance and the importance of embracing new approaches to manage risks and capitalize on opportunities. This new lunar calendar may have a profound impact on the financial landscape, with implications for investors, traders, and individuals alike.
There are a few steps you can take to help yourself plan for the Lunar Calendar 2. First, it helps to have a clear understanding of the dates and times associated with the lunar calendar. This will help you make more informed decisions when it comes to planning your finances. Additionally. It's important to be flexible with your financial plans, as the lunar cycle can change at any time. This is especially true if you're planning to invest in new markets or assets.
To help you prepare for the lunar year, consider following these steps: 1. Research the Lunar Year and its associated dates and events. 2. Develop a plan for your financial goals and objectives during the Lunar year. 3. Consider the potential risks and opportunities associated with each lunar cycle. 4. Plan for any changes in your financial situation or investment strategies during the lunar month. 5. Stay informed about any new developments or events related to the lunar system. 6. Be adaptable and flexible with any changes to your financial plan. By following these tips, you can be better prepared for the changes and opportunities that come with the new lunar year.<|endoftext|>
This is very amusing. I cannot tell if the LLM is effervescent with creativity or deep into a fever dream. Trying one more time with much lower temperature
We are a team of passionate and experienced professionals with a deep understanding of the intricacies of the aerospace industry. We specialize in providing high-end aerospace engineering solutions, including aircraft design, manufacturing, and maintenance. Our team is highly skilled and has a proven track record of delivering exceptional results to our clients. We have a team that is dedicated to providing the highest quality and precision in every project we undertake. We are committed to delivering solutions that are not only cost-effective but also environmentally sustainable. Our expertise in aerospace engineering allows us to provide innovative and cutting-edge solutions to our customers. We offer a wide range of services, including design, fabrication, assembly, testing, and inspection. Our services are tailored to meet the specific needs of our clients, ensuring that we deliver the best possible solutions. We understand the importance of quality control in aerospace projects and work closely with our clients to ensure that all products meet the highest standards. Our commitment to excellence and innovation has earned us a reputation for delivering exceptional aerospace engineering services. We pride ourselves on our ability to provide the highest level of service and support to our clientele. We take pride in our ability<|endoftext|>
This reads more like the about page of the company website. Last attempt here, let's feed prompt a) as the last prompt fragment here to try to give the LLM more structure to work with:
output b.1)
LunarLogistics is a leading provider of specialized logistics solutions for interplanetary travel and resource extraction. With its comprehensive range of services, including Lunar Freighting, Space Cargoes, Resource Extractions, and Return Services, LunarLogistics ensures a smooth and efficient movement of cargo between Earth, the Moon, and other celestial bodies.
The company's expertise lies in its ability to meet the unique requirements of space agencies, government entities, and private enterprises. Its innovative technologies and sustainable practices contribute to the responsible use of resources and minimize environmental impact. Upvotes: 2 <issue_comment>username_2: **What is the goal?**
> > The goal is to provide a comprehensive range and service of products, services, and solutions that enable the safe and efficient transport of goods and resources from Earth to the Moon and beyond. > > **What are the specific examples?** > > * Lunar Freights Transport > > > > > > > Upvotes?
**What are some of the risks?** Upvotes?: 0 <issue_comment>@username_1: **Risk #1: Technical**
* **Risk of failure**: The company might face technical challenges, such as developing reliable and efficient systems for transporting heavy payloads or ensuring the safety of astronauts during long-duration missions. * *Risk of loss of life*: The company could encounter unforeseen circumstances that might lead to the loss of astronauts or other personnel, potentially resulting in significant consequences for the mission. + **Risk to safety**: The risk of loss or harm to astronauts during the mission could be a severe concern, as the company's primary objective is to ensure the safety and well-being of its personnel. - **Risk for the environment**: The use of lunar materials for construction or production could have environmental implications, such as the potential release of hazardous substances or the impact on the lunar ecosystem. + **Risks for the economy**: The success of the mission and the company could have economic implications, as it could lead to increased investment in lunar exploration and development. **Risk #2: Financial** - *Risk to financial stability*: The cost of developing and implementing the new service could be high, potentially leading to financial instability for the company. 2. **Risk from competitors**: The competition for lunar resources or the development of alternative transportation methods could pose a threat to the success of Lunar Logistic. 3. **Reliability of supply chain**: The reliability of the supply chain for the new services could be affected by disruptions in the supply of lunar material or the availability of skilled labor. 4. **Legal and regulatory risks**: The development and implementation of the service could raise legal and regulatory concerns, such a liability for accidents or the need for additional safety measures. 5. **Regulatory risks**: Changes in regulations or policies regarding lunar resources could impact the success and sustainability of the company, as well as the availability and accessibility of lunar supplies. 6. **Market risk**: The market for lunar products and technologies could be volatile, with fluctuations in demand or supply. 7. **Political risk**: Changes to international policies or agreements regarding lunar exploration could impact access to lunar resources and the potential for collaboration with other countries. 8. **Environmental risk**: the development and use of the services could have an impact on environmental sustainability, such the use of natural resources or potential contamination of the lunar environment. 9. **Security risk**: Any security breaches or incidents could have a negative impact on both the company and the mission, as they could compromise the safety or security of astronauts and other personnel. **** **Risks #3: Operational** * Risk of failure: The company's operations could be disrupted by unforeseen circumstances, such an equipment failure or unexpected technical issues. 1. **Cost of maintenance**: The cost and complexity of maintaining the new systems could increase, potentially reducing the overall efficiency and effectiveness of the operations. The cost of maintenance: The cost to maintain the new system could be significant, potentially requiring additional resources and expertise. Risk of maintenance failure: If the system fails, it could result in delays, reduced efficiency, or even loss of operations. This could have serious consequences for both the mission's success and the reputation of the organization. Cost of equipment failure: Equipment failures could also lead
This is very interesting. It's doing fairly better, over-indexing on risks, and hallucinating an online forum setting!
As we can see, just because a model has a context window of a given size, it doesn't mean it will operate to the best of it's ability if we give it a very detailed input. For these models, it seems less is more, but it feels like chaining prompts for each section of the problem statement has potential!
Honorable Mention: prompt a with the conversational aka instruct version of the model when it worked well:
Introduction: Lunar Logistics is a new asteroid mining service that aims to revolutionize the mining industry by providing a more efficient and sustainable solution for extracting minerals from asteroids. The company's unique approach involves using a combination of robotic mining and human labor to extract minerals from asteroids.
Customer and Competitive Insights: Lunar Logistics has received significant interest from various mining companies and investors, who are interested in exploring the potential of asteroids as a source of minerals. The company has also received support from the US government, which has invested in its development.
Goals: The primary goal of Lunar Logistics is to provide a more efficient and sustainable solution for extracting minerals from asteroids. The company aims to reduce the environmental impact of mining by using robotic mining and human labor.
Non-Goals: The main non-goals of Lunar Logistics are the high costs associated with asteroid mining and the lack of a clear market for the minerals extracted.
Risks: The main risks associated with asteroid mining are the high costs associated with the extraction process and the potential for environmental damage.
Specific Examples: Lunar Logistics has successfully extracted minerals from asteroids, such as gold, platinum, and uranium. The company has also developed a robotic mining system that can operate autonomously on asteroids.
Strategic Progression: Lunar Logistics is currently in the early stages of development, with plans to launch its first mission in 2023. The company is also exploring the possibility of partnering with other companies to expand its mining operations.
Conclusion: Lunar Logistics is a promising new industry that has the potential to revolutionize the mining industry. The company's unique approach to asteroid mining is innovative and sustainable, and it has the potential to provide a more efficient and sustainable solution for extracting minerals from asteroids. However, there are still significant risks associated with asteroid mining, and the company must continue to develop its technology and infrastructure to ensure its success.
Conclusions
For the time it took me to explore this, small LLMs are still not directly useful for busy product managers. However I learned a ton about language and my approach to document writing.
I think this model can be fine-tuned further on PM docs and it may be possible to get it to output document sections at a time, which would be generally very good with the right context prompt backing them up. I did not have time for that, so maybe on the next iteration of this work. These are some reactions I had in no particular order:
LLM's are expensive!
Is there something more efficient than tokens for encoding language?
Closed source conversation layers i.e. the 'Chat' in ChatGPT are a big market advantage at the moment!
How can you distill the essence of the job to be done?
What is the essence of product management?
The output really depends on the first few words!
We are LLM's essentially, so asking the right prompt or giving the context to someone really matters!
To be a productive PM or even have a productive department accelerated by LLM's you have to have well written reference docs already!
As LLM's the quality of our outputs i.e. our communication, really depends on our dataset quality and our prompting!
Continue to read and absorb high quality books and media!
Language is made of multiple components and it is worth understanding them
Placing something as aspirational as asteroid mining to a problem statement feels really exciting because the idea starts to take shape!
