Background: this is a written long form of the presentation i gave during the ASEAN Digital Minister’s Meeting. Slides are available here

AI is slowly becoming foundational infrastructure for the next era.

The real question is no longer whether AI will be adopted, but:

• who controls it

• how it is adopted

• and who benefits from it economically

I get asked a version of this question very often:

How do we build a local AI ecosystem?
- For economic growth?
- For sovereign AI control, privacy and security?

However, there is a common mental model behind most of these discussions.
Which in my opinion, is wrong.

The wrong pipeline most people are assuming

This is roughly the path taken by OpenAI and other frontier labs.

It is also:

• extremely expensive

• slow (takes years to build up talent and infrastructure)

• optimized for research labs, not local ecosystems

And by the time applications are being built, with commercial value and economical impact. Years would have gone by.

Why this way of thinking fails in practice

The problem with this model, is not that it’s incorrect.

It’s that it assumes you must finish the earlier stages to do the later ones.

This assumption no longer holds. Open source AI has changed the economics.

For example, below - guess which response is from ChatGPT 5.2, and which is from an Open model downloadable today.

Today, strong pre-trained and instruction-tuned models already exist off the shelf. Many of them are competitive with frontier systems for most real-world tasks.

The main exception being on the “frontiers” of AI used in scientific research, and long horizon tasks. Most of which is not needed in most industries such as banking and customer servicing.

In that sense, others have already:

• paid the cost of experimentation

• absorbed the cost of failure

• released their work publicly

You do not need to repeat that journey.

Start where the real commercial value is …

Specifically:

• Applications with existing large models, using custom prompts with RAG systems, proving adoption

• Before subsequently fine-tuning to expand use cases, language support,
  and new domains to have tighter control and alignment

While we cite $10k-1M for these efforts, the later covers isolated larger projects and initiatives with large datasets. For the vast majority of projects we see, this trends towards the $10-100k cost.

That is cheap enough for:

• universities

• startups

• small teams & grassroots efforts

In addition most of the goals for Sovereign AI is to maintain

• Domestic use case & language support

• Privacy & Security

• Reliability & Control

All of these goals are achievable by fine-tuning existing major models instead of pre-training from scratch. And be flexibly deployed either in sensitive private datacenters (for secure govt use), or public cloud (for commercial usage).

Highlighting case studies : Who did not need 100’s of millions for impact

Case Study 1: Sunbird AI - Sunflower AI models

Sunbird AI released Sunflower, Uganda’s first homegrown multilingual AI model designed specifically for local languages. Outperforming Google’s and OpenAI systems in local languages use cases in 24 out of the 31 Ugandan languages.

Built by fine-tuning existing major open source models from ranging from 14 to 32 billion parameters. With models small enough to run on medium to high end notebooks.

With real-world use cases Sunbird AI is being used by : farmers for agricultural guidance in their native languages, educators can translate learning material for students, and healthcare workers can communicate more effectively with patients without language barriers.

Advancing digital inclusion, cultural preservation, and equitable access to technology across Uganda.

Case Study 2: Kissan AI - Dhenu Agriculture model

Kissan AI developed Dhenu (named after the divine bovine goddess), in inida, a language model trained for agricultural and farming use. In English and Hindi

Built by fine-tuning existing major open source models from ranging from 3 to 8 billion in parameters. Which are models that are small enough to run on mid-to-high end phones and tablets.

They allow the usage of AI, where internet connectivity is sparse or even non-existent. Helping farmers improve decision making and productivity.

Applications first, not pre-training or infrastructure first

By starting with application and/or finetuned models, we avoid a common failure mode I see is this:

“We will build a national model first, then later figure out how to use it.”

Which often burns through tens to hundreds of millions of dollars with little to show.

When applications are deployed on the ground, they create direct feedback:

• whether better language support is needed

• whether models need to be more specialized

• whether inference costs are too high

• and where infrastructure becomes the bottleneck

Achieving economical impact within months not years. While having the flexibility to move up the chain when demand for it grows.

More importantly:
Even if you pre-train it right, but do not fine-tune a model well and fail to deliver on no applications using it, you have failed — with little to no economical impact with the millions invested in.

The reality is, all ideas will carry risks, but starting from applications allows you to test economically multiple ideas at scale, quickly - at the price of a single pre-train model, or a major AI datacenter project.

Its also worth acknowledging: when it comes to new technology where one do not fully understand. Policy has a tendency on focusing on the familiar, infrastructure.

What about AI control and abuse?

For control and AI abuse prevention, these can be safeguarded through policies, that prosecute the misuse of AI. Essentially treating it like a dual use tool.

A mental model i have advised policy maker, is to write policies from the lens where there is access to large pools of outsourced low-cost labor. And apply it to both humans and AI usage.

For example:

• Policies against the usage of large number of fake/paid personas, used for manipulation of public opinion and elections. Done through the usage of large pool of human labor and/or AI agents, pretending to be domestic residents.

• Policies against the usage of false images and videos, for slandering and blackmailing. created through professional photoshop / video editing / AI image or video generation.

• Policies against strengthening cross national prosecution against scams. Done through Human / AI call centers.

The unfortunate reality, as the AI model sizes required to achieve some of these abuse behaviour is shrinking to one that is runnable on “off-the-shelf” hardware like high-end laptops. Because these use cases do not need “large 500 Billion parameter models”

Centralizing into a sovereign AI model and infrastructure. Will not make these tools disappear.

Common push back: But how are we gonna compete with the bigger models?

Here is an industry secret: for specialized production application you do not have to.

You only need to get it to be reliable and good enough for the end-customer use case. Another factor is, when you run things commercially at scale, the bigger the model gets, the more expensive it becomes.

This is reflected in our traffic data usage as well

Because individual users, tend to use AI models across a large spectrum of use cases, individuals tend to use more versatile models (they can pick any of the models on our platform at the same price), where larger models dominate their usage.

However for commercial scale, where the smaller models means “more tokens and requests” for the same dollar. We see heavy reliance of 12B to 32B models in the use cases tuned for them.

You do not need an expensive Einstein model to do everyday college task at scale. Especially when you can get more consistent and even better results, with efficient trained specialized model, for your domain.

We expect this trend of larger off the shelf models being used for exploration, and experimentation to maintain. As it balances the need for fast experimentation and go-to-market. And efficient cost management of AI at scale

One more thing: Pre-trained models are becoming a commodity

Two years ago, a strong open-source model was released every few months.

Today: releases happen monthly, sometimes weekly with the gap closing. All while running on smaller hardware, and outperforming the best close-source models one-to-two years ago.

While major closed source commercial models are still in the lead.

• The gap has been shrinking rapidly in the open model space

• While being smaller and cheaper to run. Today the best open models that can run on a $10k desktop PC can give similar or better results to the larger GPT4 models which required $300k servers in data centers to run.

The pace is so fast that several times many new pre-trained models are obsolete within days. Benefit from this trend by building on pre-trained models, instead of against it.

Few industries exists, where the best commercial solution, which costed 100’s of millions, is made freely available consistently in under 2 years.

Clarifying question: If some of these models can be deployed on cheaper hardware, why is there still strong demand for larger servers?

There are two major factors for AI, the model size, and compute.

Larger model sizes, pushes the minimum hardware requirements for memory chips in the AI system. For example, a high-end Apple PC with 256 GB vram (or 256 Billion Bytes), can run a 120 Billion parameter AI model. But not the larger 1.7 Trillion parameter model, GPT 4 alleged size. Which would requires higher end servers with 2 TB vram (or 2048 Trillion Bytes)

Another major factor is compute. While you can run on a mac studio a 120B model for one user at a time. will struggle for anything more then that. It lacks the compute compared to a high end AI GPU server, which can run for over a 100 users in parallel the same model, at faster speeds.

At scale, when serving thousands or millions of users, these higher end machines are essential. But for business who do not need such volume, open source models help them right size to their needs.

Answer to “guess the model”

About the author

Eugene Cheah is the founder of Featherless AI and co-lead of the RWKV open-source AI project, the first AI project under the Linux Foundation. He works on AI inference, efficient architectures, and building practical AI systems used in production globally.

With our background in Open Source AI - We are interested in collaborating in scaling open source AI models to meet your nations or Soverign AI needs, with your local industries. Feel free to reach out to eugene @ featherless.ai

Beyond Big Budgets: The Quiet Disruption of Capital-Efficient AI

Since 2024, the AI narrative has been dominated by a narrative: Only Big Tech, with billion-dollar budgets and GPU arsenals (Nvidia’s historic climb), could compete in foundational AI. This belief, fueled by OpenAI’s AGI billion dollar fundraising spectacles and Meta’s $65B pledges, have convinced many investors that to compete, you have to either compete at a planetary scale or not compete at all.

Yet as we enter 2025, DeepSeek’s trajectory, achieving frontier performance at a fraction of the cost, echoes several of my warnings

• The AI ecosystem needs open source & diversity to thrive

• You do not need half a billion dollars to succeed

• $2 H100s: How the GPU Rental Bubble Burst

The next decade of AI won’t be won through brute force computation alone, success will come from teams that combine technical ingenuity with capital efficiency.

We will be going through the how, and my predictions moving forward.

Deepseek comparison table with Leading Models

How DeepSeek does it: Efficiency through specialization

Deepseek’s rise to the forefront was not done overnight, instead its success emerged from a year of deliberate, thoughtful, incremental, specialization across two critical domains:

1. Specialization in mixture-of-experts architecture

2. GPU compute optimization (forged under the constraints of hardware sanctions)

This strategic focus mirrors a broader trend among Chinese tech players, from Tencent and Alibaba, to smaller startups (like StepFun), all compelled to innovate efficiency-first solutions in response to limited access to H100-class GPU’s.

Unable to match the 100k GPU clusters of Meta and other western tech giants, they have turned necessity into ingenuity, narrowing the gap with Big Tech which has been demonstrated by DeepSeek.

1) Mixture-of-experts: Scale smart, not just scaling up

Traditional transformers, like Llama, activate 100% of their parameters for every token generated. Mixture-of-expert architectures flip this script. Instead of making use of one large model, they are a network of smaller specialized models, of which only a portion of them gets activated at any point of time. Reducing the cost of compute for both training and inference.

At Recursal, we validated this efficiency firsthand with Flock of Finches: RWKV-6 MoE matching the performance of our dense model while reducing activate compute to approximately 20%. This demonstrated how MoE can amplify capability without proportionally increasing the inference cost.

For DeepSeek V3, 671 Billion parameter model, this translates to a 5% compute cost, by using only 9 out of 256 experts. Making it, despite its larger size, more efficient to train and run than even Llama 405B, which has a smaller total parameter count but makes use of 100% of its parameters.

Yet MoE’s advantages come with challenges, Training these models correctly is significantly harder to get right in comparison to a full transformer model. A trial-and-error process which only grows more complex with each added expert.

Which is part of the reason for DeepSeek’s iterative process from 64 experts in its 16B Model (Jan 2024) scaling up to 164 experts in v2 to 256 in v3 today with improved efficiency in each iteration.

Resulting into successful implementation’s of MoE systems with several orders of magnitude higher expert counts then most western labs (typically at most ~8 experts)

This reduction is what allowed them to reduce the training cost from over $100-50M in traditional transformers training to under $10M with MoE.

2) Optimizing GPU efficiency: Squeezing compute from sanctioned GPU lemons

Another major advancement made, goes beyond how the model was designed at a high level and focuses more on extracting maximum value from constrained resources and GPU utilization.

While we can go into deep technical details on each one of them…

Its less about a single innovation, but compounded improvements made where they transform sanctioned H800 clusters into 2x-more-efficient workhorses than unrestricted H100 systems running conventional AI frameworks.

Some of these techniques were developed and have been tested over time in earlier versions and were mentioned as a contributing factor. When layered atop MoE’s 5-10x cost reductions, the total efficiency multiplier reaches 10-20x.

We intentionally provide a range, to avoid debating the accuracy of numbers, focusing more on the order of magnitude of change.

Less a sudden revelation, more of an incremental build up

For the past 1 year, the top 2 teams in China have been incrementally making all these improvements, while iterating on their own respective techniques over multiple models both big and small over time. With gradual increment at every step. All while keeping training costs to a low budget.

The end result was more of an incremental upgrade every 2 months approximately.

Over the past year, China’s two leading AI teams have pursued a disciplined strategy of continuous improvement, refining their distinct methodologies across models of varying scales, big and small. By adopting a gradual approach, they implemented steady, measurable improvements at each development cycle, all while operating with budget constraints. This iterative process culminated in a pattern of consistent progress, with upgrades getting rolled out approximately every two months.

So it was a surprised for no one who has been watching. But a surprise for everyone being ignorant in the progress outside of the big labs.

Prediction 1: Big labs loses moat of scale

The key takeaway from Deepseek's success is the clear evidence that we do not need hundreds of millions in compute budgets to train O1-level models. While there may be some reporting discrepancies in the actual capital and resources spent, Deepseek's story demonstrates that the effective cost of building and training models could be significantly lower than what the major players—Meta, Microsoft, and Google—would have us believe.

These tech giants consistently push the narrative that model development is an overwhelmingly capital-intensive endeavor, creating a perceived barrier to entry for smaller players. Given the resource intensive nature of large transformer models.

A narrative the market adopted, in assuming that the resource-intensive development pathway is the only way to lead in this AI race.

While members of these companies, actively advising investors, against investing into smaller alternative startups.

This strong belief, leads to strong resistance of ideas, that challenge the notion. Stifling innovation that would have came from smaller nimbler teams (A belief which fools not only themselves, but their investors)

While at the same time, this benefits the larger players as they become the concentration of AI investments, and pushing their valuations upwards into trillion’s of dollars. Into an oligopoly.

However smaller companies like Deepseek and open source project alternatives to transformers like RWKV (which I work on) challenge the notion that AI development demands insurmountable resources.

As they focus on efficiencies, and lowering the barriers of entry.

Open model builders are rapidly positioning themselves as attractive investment vehicles for institutional investors. By offering a compelling alternative to the centralized dominance of big labs, they enable investors to diversify their bets and reduce over-reliance on a handful of AI giants.

This shift not only democratizes innovation but also creates a more competitive and resilient AI ecosystem. With alternative investment opportunities.

Prediction 2: NVIDIA’s hardware monopoly is cracking and that’s great for everyone

The industry stands at an inflection point. What began as NVIDIA’s unchallenged dominance in training infrastructure is rapidly evolving into a more nuanced landscape where inference capabilities take center stage. This shift isn’t merely technical, it represents a fundamental restructuring of the AI economy. While training does remain critical, new players e.g. DeepSeek allow smaller clusters to train models competitively, eroding the necessity for hyperscale infrastructure. NVIDIA’s historic premise of bigger clusters equal better results is showing its cracks.

As enterprises deploy AI in their organizations and applications demand for dedicated inference platforms is surging. Challengers like AMD, Groq and Cerebras are capitalizing here, offering tailored solutions where alternatives perform as well or even better.

For developers and investors, this spells opportunity as the market is no longer bottlenecked by scarcity but instead a more dynamic and accessible economy where innovation can thrive in abundance.

Prediction 3: Raise of Open models

Open-models will drive faster iteration and innovation than any single corporate lab by unlocking community collaboration. Meta’s Llama models demonstrated this potential but DeepSeek’s MIT-licensed R1, which open-sources both its model weights and technical innovations. Democratizing AI accessibility to more companies and communities.

Its not just llama, deepseek, or qwen, and a whole new wave of open models which will be competing with each other. With the commercial incentive on adoption for their API platform (qwen) or consumer applications (llama) or both (deepseek).

To accelerate the growth of open-model development, resources and capital must be made accessible to as many contributors as possible. However, one of the biggest pain points for any AI developer that develops his own application today is “How can I run and test the latest open-models without the pain and cost of setting up my own infrastructure?”

Here is where platforms that handle the complex infrastructure (GPU management, model deployment, access to a wide range of models) become a necessity in the development cycle.

These platforms are doing for AI what cloud computing did for web development by simplifying the technical overhead and reducing the costs by 10-100x for any developer. Just like what platform’s like featherless.ai, which we built at recursal, have done.

As AI continues to reshape industries, the platforms that makes open source power more accessibility represent a strategic opportunity for forward-thinking investors to capitalize on the expanding ecosystem of open models.

Closing words

Investors betting on AI’s next decade must ask: Are we funding another GPU arms race led by Big Labs, or do we look at open ecosystems where efficiency and ingenuity coexist. The next decade of AI won’t be won through brute force computation alone, success will come from teams that combine technical ingenuity with capital efficiency.

While capital will always play a role, DeepSeek’s achievements signal a fundamental shift in the opportunity landscape. The most promising investments may not be in replicating existing approaches at scale, but in identifying teams that can leverage efficiency innovations to create outsized impact with limited resources.

We’re entering an era where innovation and efficiency aren’t just nice-to-have features, they’re essential competitive advantages that will determine which projects succeed in bringing AI’s benefits to the broader market.

If you’re an investor interested in exploring these opportunities or a team building with these principles, I’d welcome connecting to discuss how we can collaborate in shaping this more efficient, accessible future. Reach out to me on linked in or twitter (@picocreator) to start the conversation.

🖖🚀
Eugene
CEO @ recursal.AI

Acknowledgment & References

Several parts of this article, including the research was done by the content team at featherless

Data points are sourced from
- https://arxiv.org/pdf/2412.19437v1
- openai.com/index/gpt-4-research/
- wired.com/story/openai-ceo-sam-altman-the-age-of-giant-aimodels-is-already-over/
- api-docs.deepseek.com/quick_start/pricing
- help.openai.com/en/articles/7127956-how-much-does-gpt-4-cost
- https://arxiv.org/pdf/2407.21783
- apxml.com/posts/training-cost-deepseek-v3-vs-llama-3
- together.ai/pricing

This is follow-up footnotes - for the previous viral article, originally posted on latent.space. That didn’t make it into the original article (too much math).

This is meant to be a “living post” for any future amendments as well.

It is recommended reading this first for context:

Latent Space

$2 H100s: How the GPU Bubble Burst

Don’t miss Eugene’s responses on the HN and Reddit and Twitter discussions…

Read more

2 years ago · 55 likes · 11 comments · Eugene Cheah

My spreadsheets for calculations can be found here

• new H100 ROI spreadsheet (Aug 2024)

• H100 Infiniband Cluster math spreadsheet (Aug 2024)

Not all H100 clusters are made equal, broadly speaking they are segmented into the following bands, decided by their, InfiniBand.

• <= 2 nodes: Direct Infiniband interconnect (pretty much only unis)

• <= 64 nodes: 1 layer of Infiniband switches (common small clusters)

• <= 2048 nodes: 2 layers of Infiniband switches (most common size range)

• <= 65,536 nodes: 3 layers of Infiniband switches (handful, big players only)

H100 Training clusters are extremely network-hungry, due to the nature of the training process, and are typically deployed with 3.2Tbps Infiniband interconnect. Built using 8 x 400 Gbps connections.

The problem: as the cluster size increases, the more complicated this network becomes to support the levels of interconnectivity required.

This as a result has a direct impact on the unit economics.

PS: 100k H100’s is the approximate cluster size for x.ai cluster

Where each tier of the cluster node sizes (<=64, <= 2048, >2048), has a bump up in the H100 node CAPEX economics; Ranging approximately between $47k to $52k per H100. And why I used the $50k average previously.

Additionally, due to the sheer energy scale of these GPUs. Where each H100 server, consumes as much energy as 4 US households. The numbers projected here are conservative.

For small clusters (<=64), these can possibly be retrofitted in existing data centers.

For larger clusters, the energy consumption starts to rival that of Small or even medium-sized cities. Dedicated cooling and power lines, or even generators will be needed. Dramatically, increasing the setup unit costs involved.

Combined with the high failure rate of H100 (they have <99.9% uptime), Nvidia themselves project a 1:1 ratio for the cluster facility CAPEX & operation costs. So we are being very conservative by using only 50k per H100 node instead.

I will not be surprised if the real number is easily 1.2x higher. Especially for larger clusters, due to how price inelastic H100 servers are when buying from nvidia.

Shoutout: Semianalysis related write ups

These folks are among the best in the industry, for these kind of analysis. And is worth reading, if these kinda of infrastructure deep dives is your thing.

Answers the question on: How much will the setup cost be (minus facility), when you do custom networking solutions instead of following nvidia reference

SemiAnalysis

AI Neocloud Playbook and Anatomy

The rise of the AI Neoclouds has captivated the attention of the entire computing industry. Everyone is using them for access to GPU compute, from enterprises to startups. Even Microsoft is spending ~$200 million a month on GPU compute through AI Neoclouds despite having their own datacenter construction and operation teams. Nvidia has heralded the rapid growth of several AI Neoclouds through direct investments, large allocations of their GPUs, and accolades in various speeches and events…

Read more

2 years ago · 103 likes · 9 comments · Dylan Patel and Daniel Nishball

H100 networking, and failure rates, in even more details

SemiAnalysis

100,000 H100 Clusters: Power, Network Topology, Ethernet vs InfiniBand, Reliability, Failures, Checkpointing

There is a camp that feels AI capabilities have stagnated ever since GPT-4’s release. This is generally true, but only because no one has been able to massively increase the amount of compute dedicated to a single model. Every model that has been released is roughly GPT-4 level (~2e25 FLOP of training compute). This is because the training compute dedicated to these models have also been roughly the same level. In the case of Google’s Gemini Ultra, Nvidia Nemotron 340B, and Meta LLAMA 3 405B, the FLOPS dedicated were of similar magnitude or even higher when compared to GPT-4, but an inferior architecture was utilized, resulting in these models falling short of unlocking new capabilities…

Read more

2 years ago · 170 likes · 22 comments · Dylan Patel and Daniel Nishball

Amendment: in electricity cost calculation

Shout out to and @youjiaccheng for spotting the minor math error, which has been fixed

Featherless.AI plug …

What we do …

At Featherless.AI - We currently host the world’s largest collection of OpenSource AI models, instantly accessible, serverlessly, with unlimited requests from $10 a month, at a fixed price.

We have indexed and made over 2,000 models ready for inference today. This is 10x the catalog of openrouter.ai, the largest model provider aggregator, and is the world’s largest collection of Open Weights models available serverlessly for instant inference. Without the need for any expensive dedicated GPUs

And our platform makes this possible, as it’s able to dynamically hot-swap between models in seconds.

It’s designed to be easy to use, with full OpenAI API compatibility, so you can just plug our platform in as a replacement to your existing AI API for your AI agents. Running in the background

And we do all of this; As we believe that AI should be easily accessible to everyone, regardless of language or social status.

Additional Sources:
- ChatGPT launch date: Wikipedia
- H100 launch date: Tech Power Up Database
- The A100 SXM had 624 bf16 TFlops, the H100 SXM was 1,979 bf16 TFlops
- Microsoft & AWS allocated over $40 billion in AI infra alone: Wall Street Journal
- “600 Billion Dollars “ is about: Sequoia’s AI article
- Nvidia investor slides for Oct 2014: page 14 has the pitch for “data centers”
- Semi Analysis: deepdive for H100 clusters, w/ 5 year lifespan approx for components

🇸🇬🇸🇬🇸🇬 National specific content 🇸🇬🇸🇬🇸🇬
The following article was specifically written for Singapore audience where I am born from. And may or may not be applicable to my wider audience (hi to the rest of the world)

Additionally, because this is targeting a Singapore audience, we will be using the local terminology of AI Creator (Foundation model makers), and AI Practitioners (More commonly known as AI Engineers)

It also comes in two parts
- Part 1: What does it means to be an AI hub?
This is meant to be a primer for a more general audience, if you are not a startup founder, it is highly recommended you read this first.
- Part 2: A blueprint for turning Singapore into an AI startup hub.
(this article)

Previously I asked the fundamental question…

And if the answer is to be a vibrant startup AI hub, this article is my answer.

I will cover the

• advantages we have

• the problems we have

• case study of Auto-code-rover vs Devin

• how France pushed back (and their basic blueprint)

• the roadmap, how can we replicate what France did

• the threats Singapore face and why I am writing this

What Unfair Advantages does Singapore have in the global AI space?

Here is the crazy thing:
Singapore literally have all the infinity stones to make this happen - if we can put it all together, and “Snap” it into place.

Having spent a good 9+ months in SF the past year. The following is my observation of Singapore's unique strengths that we have in AI.

Unmatched by almost any country besides possibly Canada, and France.

1. We have the GPU compute, to support such AI projects

   1. The basic GPU infrastructure for H100s is present locally in Singapore. Especially for the fine-tuning and production deployment phases.

   2. We have the capacity even for foundation model training if needed. Though as outlined in the previous article. Foundation model training capacity does not need to be colocated in Singapore. It is perfectly alright to build the teams here and train overseas (like in Finland)

2. We have the talent pool for this in SF, and the raw talent in Singapore

   1. the typical AI team in SF, has disproportionally more Singaporeans among them than any other individual country in Asia. (Japanese, Malaysian, Koreans, many cases even India and China) - we are ridiculously over-represented

   2. It is a mini-meme in SF AI circle, to spot the Singaporean (Are you Singaporean?) - the record holder has “spotted” me in under a sentence. That’s how prominent we are.

   3. This talent pool has been built up over time, fortunately, or unfortunately due to the talent drain that has been happening in the SF, due to their cultural insistent on a local presence.

   4. a large part of this talent pool has a strong desire to return - if an equivalent career opportunity presents itself

3. We have Singaporeans who have a strong influence in the AI/devrel space

   1. The number 1 AI influencer in SF, and what Silicon Valley has dubbed the king of devrel - due to how he coaches other Big Names respectively;

   2. Shawn or more commonly swyx in the US. Is Singaporean, and is constantly supporting Singapore founders in SF, providing them with critical guidance on how to grow their business in the SF market.

   3. Many other Singaporean founders within the SF community

4. EnterpriseSG has established supporting infrastructure to facilitate Singapore companies to commercialize and/or raise funds in the US market.

   1. In particular through their partnerships with accelerators like 500 startups

   2. Singapore Free Trade Agreement permits founders to freely travel back and forth in 90-day intervals, for business purposes. A process that would have required other countries to have a lengthy visa approval process - Without this, many of the AI startups that Singaporean founders are building in SF would not have been possible.

   3. This allows us to move freely back and forth and makes it possible to build in SG and sell in the US.

Some of these are achieved by coincidence, and some of it is achieved through the hard work of multiple individuals in existing government agencies.

From infrastructure, people, capital, and GTM. We have all the basic building blocks in place. Building blocks, that multiple nations will need to spend hundreds of millions to develop. That will be able to help build a pipeline for success.

The problem Singapore founders face …

However, it’s not without flaws… There exist 3 major issues that Singapore founders face when trying to build an AI technology.

1. Lack of Customer demand

   1. While it is growing, due to the lack of maturity and adoption of the AI scene locally and in Asia. The current demand for AI in the region is insufficient to sustain the average startup. As such, founders turn to the US market instead to address their revenue needs.

   2. This is a combination of both smaller market sizes
      (compared to the US, Europe, India, and China)

   3. And how Asia in general, is further behind the adoption curve
      

2. Lack of Support, both financially and on the local front

   1. Local investment is hesitant to support local founders in AI, rightfully so due to the challenges faced with “lack of customer” demand. Hence investment has historically been into segments that have known success locally (eg. e-commerce, fintech)

   2. This has cascaded to Enterprise business purchasing sentiments itself.
      As experienced in my previous startup (uilicious):
      we made more enterprise sales progress locally, by presenting ourselves as a US startup based in SF expanding into Singapore. Then selling ourselves as a Singapore startup to Singapore enterprises.

   3. Our enterprises locally are spending orders of magnitude more for overseas AI solutions, than our total investments locally. So not only do local startups have to fight for tighter funding, but they also have fewer resources to win deals locally.
      

3. Pressure from SF investors, to concentrate all assets in SF

   1. There is a strong conviction that the best place to build AI is in SF itself from the investors, in hiring talent, selling to customers, etc. And ignoring the cost of living in SF, they are mostly right.
      
      However what makes this more difficult, is how often it is for multiple funds, that this implicit requirement, becomes a “dealbreaker” condition that is not always verbalized out loud.
      

   2. Really large SF funds operate with a high-risk high-reward mindset. They accept failure as part of the process. And operate at scales, that rival governments of smaller nations.
      
      For such VCs, by concentrating the human capital in SF, they can help partially mitigate the risk of failed startups, by folding up the talent into their successful startups.
      
      Most startups fail due to market or tech reasons, or mismatch of human talent. Not the lack of talent in itself. As such, it is extremely common for VCs to match the talents, of either exited founders or failed startups. To their existing portfolio hiring demands, be it for their bigger winners, or their newer “hot startups”.
      

   3. (Eugene @ recusal): I am experiencing this first hand myself, both the pressure and the benefits - the past 3 hires from the valley, have been ex-founders / operators - matched by my VCs, from recent exits and/or company closures.
      In a way, they are creating that rotating pool of talent locally, within their own funds by themselves. A goal, that is traditionally aimed for by the government in a city. And an advantage this is unmatched anywhere else today - provided you have the capital.
      Where the VCs allow their losing bets, to strengthen their winning bets.
      
      VC’s want all the talent to be in SF, for the exact same reasons why the Singapore government would want all the talent in their startups to be within Singapore locally - to grow and sustain the local economy - even if it’s for very different reasons.
      

   4. We may at times pride ourselves on being more cost-effective locally. But if the local investment scene is not willing to keep pace. Founders will be forced to follow the investment dollar within the US instead, and move the team with them - Unfortunately, in many cases, a local team outside of SF is not an asset in the fundraising process, but a liability blocking the raise.
      

These 3 major points are not unique, they have been echoed independently by nearly every Singaporean founder who has gone over to raise in the US.

In addition, Singapore founders are not blind nor foolish, we do see and keep track of the trends, and we do see where we stand locally, and in the US.

Feel free to cross-validate with Singaporean founders in the valley, for the above 3 problems.

The curious case of “NUS’s auto code rover” vs “SF’s Devin”

Because how most startups differ in various details, it can be easy at times to dismiss the problems faced above. We can always hand wave it as, differences in tech, timing, or target market. There is always that detail

However, nothing contrasts the problem stronger between “NUS’s auto code rover” project, and SF’s “Cognition Lab’s: Devin project”. Because it highlights exactly the vast difference in the network effect that SF has in itself, above anything else.

• https://github.com/nus-apr/auto-code-rover

• https://www.cognition-labs.com/

It was one of the rare examples where we had two competing projects launched within a week of each other. Working on the exact same problem.

At the start of April. With the Singapore team taking the lead in performance. Reducing the differences in many factors between them.

Both build on existing work done on the “SWE-bench” project.

While “AutoCodeRover”, outperformed “Devin”, and took the lead on the world stage. The support and networking effect of the SF scene shows the contrast in full display subsequently.

Within the next 3 weeks, the SF Devin project managed to get

• 21 Million US Dollars in funding

• Over 190,000 YouTube videos coverage (proxy to social media coverage)

  

• Over 7,000 news articles (including multiple founder spotlight articles)

In contrast, the NUS team has secured the following in the same time frame

• No additional capital raised

• 800+ YouTube videos (proxy to social media coverage)

  

• 1 News Article

While the Devin team can be argued to have a much better network connection, within the valley. As the reason for their success.

That’s exactly the point, that the various founders within the SG-to-SF community have been saying. It, unfortunately, matters more about your voice, presence, and network within the valley, than the technical results you can achieve.

We see this pattern repeated multiple times across multiple startups, AutoCodeRover vs Devin, is just a distillation of the problem.

Networking triumph merit - is a very uncomfortable statement to be saying, coming from a country that prides itself in meritocracy - And an uncomfortable truth that collectively drives all of us to SF.

This is applicable to me for recursal.ai / RWKV specifically as well…

Within a month in 3 months in SF, i have gotten into more podcast, twitter spaces, discussions with top AI influencers promoting my product at the world stage.

Then I would have gotten in a year within in Singapore. There is a direct impact to marketing, user adoption, etc. Just by “being there”

How France pushed back against SF - and built its ecosystem around “champions”

How did France work around US VC pressure to constantly move talent to the SF?

They pushed back against US investor influence, with their own investors, backed by the national funds respectively. More specifically …

• 5 Billion Euros being planned into funds for AI investments over 5 years

• 500 Million Euros to start the process of funding AI “Champions” (June 2023)

  

• Co-investment with Mistral, the AI champion for France (dec 2023)

  

Mistral investment of USD 415 Million, has approximately half its investment from the French government-affiliated funds. With the remainder being backed by US-based funds.

This allowed them to build a substantial portion of their development office in France, with a smaller office in San Francisco itself.

This let them spearhead commercial and marketing traction within the US market while starting the process of developing the European market alongside the government.

With Mistral in the center as the “AI Creator / Foundation Model”, they then conducted hackathons, workshops, and events. To encourage “AI Practitioners / Engineers” to build on Mistral respectively. While simultaneously working alongside other investors, to fund startups that were built in the process.

This resulted in a follow-up of over USD 700 Million dollars in joint investment. In the past 12 months, to AI companies (besides Mistral). Co-invested along the French government-affiliated funds.

All essentially putting a large percentage of their team to build locally in France, to export and sell to the US market, and the local European market.

This has worked well for both Macron politically, and economically, as France is now well regarded as the leader of the AI space in Europe. Something they are increasingly taking pride in both the European stage, and the local elections.

Disclosure of conflict of interest:
As I go into the specific of what can be done, while being a potential benefector and recepient of it. There is a major conflict of interest.
However the roadmap listed here does not change. And I still will standby this article regardless.
I intentionally include a competing company in the roadmap. Because I want Singapore to succeed.

How Singapore is luckier than we realize

As mentioned in the previous article. There are less than 50 AI Engineering leaders who can build a team and foundation model from scratch.

Getting 1 in your country is already harder than striking 4D. Getting 2, who is extremely capital efficient, is even harder and luckier than we realize.

Getting 2, while having all the right Unfair advantages in place to kick-start the process? Is like assembling the infinity stones.

We can essentially replicate what France and Mistral did, at a much larger scale, without spending significantly more than what France did.

Due to Singaporean practical nature, and frugality (or kiasu-ness), we somehow landed on two Singaporeans spearheading their own companies with impressive performance numbers on the world stage, at a fraction of the cost of any other Foundation Model company.

• Recursal AI: Led by Eugene (me), leading the charge in multi-lingual performance, with a multi-national effort under the Linux Foundation, building both an AI model and dataset, in collaboration with data providers in every country and language.
  
  They managed to build one of the world’s best 14B multi-lingual models, with a budget of under $2.5M in capital raised.
  

• Reka AI: This is the first ASEAN company to reach the GPT4 level of performance, with multi-modal support. With their frontier large model (unspecified size). With less than $60M raised.
  
  In contrast, the initial Open AI, Initial GPT4 training run, took $100M in GPU time to train alone. And they triumphed over them.
  
  There are less than 10 companies on earth, today who have crossed this line, and Reka AI - is the only one who did so with under a $100M budget.
  
  The fact that we are not heavily backing them yet. Is honestly ridiculous.
  

Today, there are less than 50 people on this earth. Who has the full knowledge, expertise, and experience, to build and train a 7B and higher class foundation model?

From dataset preparation to architecture to the training process. And lead such a team in the process.
That is fewer people than the number of athletes in the Olympics.

And by luck, we have 2 groups being led on the world stage, by a Singaporean. Both with an unmatched capital efficiency, which lets them compete with teams 10x their funding or more.

With how limited talent is in this scene, it would be extremely hard to replicate these 2 groups by design, even if we had the capital to do so.

So how do we make use of this opportunity in time then?

For context, the typical foundation model company, especially in the SF market, raises USD 400M and up. With over 20 foundation model companies in SF to date.

The roadmap for an AI hub in Singapore:
Replicating the mistral effect in Singapore, at 4 x scale, at the same price.

1) Set up a 500M fund, or a budget - to back “Champions” within the AI space.

Be willing to lead the rounds, to solve the “Chicken and Egg” problem. While aiming for a 50:50 split between private investors, and government capital.

Very broadly the goal should aim to fund in some combination the following. Prices are for the Government Affiliated Capital allocation (aka the 50%)

• 2-5 foundation model companies (100M total, 15 - 50M each)

• 10-20 AI Engineer / Practitioner tooling companies (100M total, 5-10M each)

• ~100 AI application companies (300M total, 1-5M each)

2) Start with “champions” who are willing to set up an office locally and play an active role in inspiring others locally.

This has to be “AI Creators / Foundation Models”, as they are the only groups who will be heavily incentivized to align with local pushes to adopt their models. They directly commercially benefit, from the growth of the local ecosystem.

Additionally, they will be the only group, willing to pay to make an AI model, that supports Asian voices, while promoting the adoption of their model. Mistral did it for french, we can do the same for Asia.

Having a strong foundation model which supports Asian languages, is also a hard prerequisite for increasing AI adoption and usage in the region, be it locally in Singapore or ASEAN. Because if the model does not support our language, it cannot be adopted and used by the people in ASEAN in our language.

Finally, as these groups would also be helping bootstrap the sales pipeline into the US market, they would also need to be able to establish their own presence and sell to the US, simultaneously to ensure they remain commercially viable.

3) Fund startups that are built from - accelerators, hackathons, workshops, and co-development events alongside the foundation models

Be willing to start funding unique use cases/ideas/products that are built. A US accelerator in Singapore, alone will not magically create a market to sell AI products to, nor will it magically create investors in Asia willing to invest. But they will help bootstrap the creation of such companies.

And more importantly, as these companies get investment, a clear signal to those who work in AI, is that it is possible to build, and get funded locally.

Support teams who are willing to build up their team here locally with 5 technical members or more (sales, marketing, and HR do not count)

It is more important to break the local perception or cycle that there is no funding here. Encourage practitioners to push the boundaries, and build out new use cases or ideas.

Promising projects from these events (or universities), should be encouraged to build their own startups, with funding. This should ideally be split between “AI tooling” and “AI applications”. For different reasons.

1. “AI tooling” companies (eg. langchain), co-sell tools alongside the foundation model creators, and help scale up the hackathon and event organizing process that happens locally. This is how Silicon Valley is able to achieve having multiple events per day. They generally build tooling to help support “AI applications” use cases, and “AI engineer/practitioners”
   

2. “AI applications” companies, sell to end users, and companies their solutions. This will have various shapes and forms. Because we are barely scratching the surface of the potential applications of AI even in SF (let alone Singapore), this is one of the biggest areas for experimentation of new ideas. Their active involvement locally will also help develop demand locally from the enterprises and the region, in the adoption of AI.

4) Leverage the media

You want to inspire locals, to the idea that they can build a successful AI startup here. Leverage the media to encourage the same goal. Allowing us to view ourselves as leaders in Asia, to develop and sell to the Asian market.

Something which we uniquely can do, by engaging the local media.

We need to grow the national story, to grow the adoption of these solutions locally. (instead of defaulting to the West)

5) Connect these growing waves of startups to sell into the US market

To help ensure the success of these startups as they scale, especially given how limited the Asian market presently is.

Connect these startups, through established programs (eg. EnterpriseSG 500 startups program), to help grow these startups, bring them to the US market, and sell there.

Let them set up an Engineering hub for solutions engineers (who connect directly with customers) while encouraging them to retain their core research and development team locally in Singapore.

This is typically after their initial round, where they have the first version of their product built, which they would be actively selling in both the US and SG markets.

In addition, once this active pipeline of “build here, sell overseas” is done. Engage the AI creators and practitioners in SF, to encourage them to build from Singapore instead.

6) Be prepared for failures, and match-make talents accordingly

Failure is an inevitable outcome in startups as you scale. It is a natural part of the process, and we should assume a failure rate of 25-95% and up for each company.

However if done right, the beauty of this process is by the end of the next year you will have approximately

- 500 AI practitioners from the AI applications companies (5 each)
- 150 AI practitioners from the AI tooling companies (10 each)
- 45 AI creators/practitioners from the Foundation model companies (15 each)

600+ AI creators/practitioners, either within maturing startups, or are leaving startups into enterprises, or are training other AI practitioners within the industry.

All within a year.

If we want to build a talent pool of 15,000 practitioners in 3-5 years, we realistically need

- the first 600+ in a year
- to train the next 2,000+ practitioners
- followed by the next 4,000+
- and the next 8,000+ in subsequent years.

We would also need policies, or schemes in place to encourage such talents to be retained locally as well. Be it through successful startups, or Enterprises, to help nature the community locally (like what happened in SF)

Additionally, a perspective that is unique to the government.

Assuming an average income of 100k++/year and a tax collection rate of 10%. When scaled up to 15,000 AI practitioners. We are looking into a $150M per year in income tax.

Meaning that within a decade, this program which helped bootstrap the entire ecosystem, can make its return in investment. And this is before accounting for the additional economical benefits and taxes through adjacent industries improvements.

All while hitting the goal, of 15,000 AI practitioners.

7) Setup a follow-on 500M fund

Investors take time to gain confidence in an ecosystem, startups, and products take time to mature. Industries and customers locally take time to mature.

Over time, once the first wave of 100+ startups is finished, switch over to let the investors lead the round while taking small steps back, while still co-investing.

The reasoning behind this, is the original 500M and this 500M fund, should be treated in part as a budget to bootstrap the industry. Its KPI / goals should be measured in terms of companies built, sustained, and AI practitioners trained in the process.

It is very unlikely it will fully make a profit in itself, within a year, and may at best do so within 6 years with a successful billion-dollar exit among the few startups / IPO. Which is rare in itself.

But once the industry is ready, maturing and self-sustaining. We can then switch over to more traditional fund investment schemes and instruments (eg. Temasek) which are profit-driven in nature.

The follow-on fund is meant to help de-risk and transition the process for these startups, from government-backed funds, to profit-driven VC funds.

And with that, you have your basic AI industry in place, ready to grow year on year. And play an active role on the global stage.

As for the specifics of how such a fund is set, that will be an exercise of creating financing between Temasek, EDB, and the various agencies (Singapore has many pockets, budgets, and schemes you can adapt and figure this out)

With all of these outlined, the fundamental question is back to the title ….

Do we believe we can be an AI leader on the world stage? Beyond words, do we truly believe in ourselves?

In America, and among startup founders, there is a concept called Manifest Destiny.

I do not mean it in the religious sense but in the conceptual one.

It is easy to sit in the armchair, and constantly doubt oneself into inaction.
There are probably 101 gaps in the roadmap I suggested (I am only 1 person)

But the ones who move the future forward, are the ones who have the vision and the ambition to build the future they see and manifest it into reality. Are the doers.

They may not have all the answers today, but they will do whatever it takes, to bring their ideas into reality. They may face obstacles and setbacks along the way. But they will keep working their way towards it, one way or another.

So my question to Singapore, to policymakers, investors, and as a Nation.

Do we see ourselves as only a Tiny Red Dot, who is unable to compete on the world stage, a constant follower? Despite all the unfair advantages we have. Or …

If you believe in the future envisioned, you will make it happen.
If you do not believe in the future envisioned, you will prevent it from happening.

We have every core ingredient in place to start making this change for our future.

We are just missing one last ingredient at every level:
The belief we can be bigger than a Red Dot.

The threat faced by Singapore if we fail to adapt in time

The threat of AI is truly an existential threat to Singapore like water or any other major forces.

In the sense that the Singapore economy consists of 3 major pillars

• Imports and exports of goods through our port

• Oil refining services

• Service industry, especially financial services

The concern in the upcoming decade would be two-fold

• The global shift away from oil due to global warming

• The move towards highly automated and augmented service industries

While the shift away from oil might be inevitable. For Singapore whose core assets is our Human talent and capital.

Once the service industry is taken away from us, by other countries who move ahead in automation and augmentation with AI. It may never come back to us.

The fear and concern is the what if? What if, we do not adapt fast enough, and get left behind in this new paradigm of a highly advanced and automated AI-augmented workforce? Where we lose a pillar of our economy.

And that we only recognize the threat, and the solutions, too little too late.

This is before we throw in the more complicated national security concerns, on how AI is currently a proxy “cold war-like chess contest” between US and China. And how having independent operational capacity (as opposed to being dependent on US or China), will help us avoid taking sides in this contest.

Why I am writing this?

1. I am incredibly concerned about how the future global economy will be in this strange new AI future, where a large % of work can now potentially be automated away by more and more powerful AI. More so than any bigger existential risk.

2. I am concerned, regarding Singapore's placement, in this new AI future, on how we may be extremely ill-equipped to handle this future change. Where services one of our core economic pillars will be automated.

3. I believe in building AI that is multi-lingual and understands our Asian voices. My US investors do not entirely believe the same. And would like Asian funds to help balance out such investor interest. (In the end, I have a professional obligation towards my investors, even if I have personal misalignments)

4. I have personal reasons, in the sense I would like to spend more time in Singapore, with my parents and various friends. Build a big part of my team here, instead of in San Francisco (as per my investor demands)

Me and my wife had plans, with our BTO in Ponggol this year. All of which have been put on hold. Also frankly San Francisco is one of the last places on earth to raise a kid from.

In a way, I want to be here in Singapore and help build up the hub here in Singapore for personal reasons.

The question for me is, does Singapore want to, help me stay here?

Signed Eugene,
From San Francisco

🇸🇬🇸🇬🇸🇬 National specific content 🇸🇬🇸🇬🇸🇬
The following article was specifically written for Singapore audience where I am born from. And may or may not be applicable to my wider audience (hi to the rest of the world)

Additionally, because this is targeting a Singapore audience, we will be using the local terminology of AI Creator (Foundation model makers), and AI Practitioners (More commonly known as AI Engineers)

It also comes in two parts
- Part 1: Explores more on what it means to be an AI hub, and is meant to be a primer for a more general audience, before part 2. (this article)
- Part 2: Goes in depth, into a blueprint for turning Singapore into an AI hub

With AI being a buzzword, much lately has been discussed about turning Singapore into an AI hub.
With the national AI strategy goal of building over 15,000 AI creators and practitioners. We must be clear about what we mean when we say AI hub.

To do so, we shall explore the various hubs that exist today. In the AI world.

But before that….

Let me correct 3 common AI scaling misconceptions…

Because AI is a completely new field to most people, it would help if I outline a few major things, that are commonly misunderstood. And it is typically all about scales.

Common misconceptions that commonly baffle, everyone else both at the highest level of government, to individuals outside the AI field.

And it’s all about scales …

1) AI Creators are rarer than Olympian Athletes.

There are approximately less than 50 AI engineering leaders out there today - who can make a new large language model from scratch.

With less than 1,000 Individuals worldwide who can contribute to such teams. Globally.

So to start it off - if the plan was to have 15,000 AI creators in Singapore, that is unrealistic at best, because that would mean bringing the entire AI creator global talent pool into this island. (it definitely can be an aspiration if we choose to do so)

Thankfully, it seems like the people behind the National AI Strategy are at least aware of this constraint.

However, I bring this up - as this rarity, is a blindspot for, any investors, or policymakers. Because it can be too convenient to think in terms of “Just get some engineers, data, and GPUs”.

To build an AI model that is larger than 7 Billion Parameters today correctly, across more than a Trillion tokens (Which I would use as a loosely defined bar for a “Barely usable foundation model”). Without burning away a million dollars or more in waste.

Would require skillsets from various domains, from dataset to model architecture, to training infrastructure. And due to the scarcity of talent in these domains, and a further scarcity of individuals with cross-domain knowledge and team leadership experience.

Who can lead a team, and create a foundation model project from start to finish?

Overall, we are talking about: 0.00000633% of the world population for AI engineering leaders. Fewer AI creators, than the number of athletes competing in the Olympics today.

In that lens, it is arguably easier to win a 4D lottery, than to find and recruit such individuals, to build and lead teams locally in Singapore.

The reason for this scarcity comes from two major reasons

• The high cost of doing experiments at such sizes = High cost to learn

• Lots of design and parameter choices, from dataset to model. The lack of scientific historical data, today means a decision has to be made in equal part “artistic decisions” and “scientifically data-driven informed decisions”
  

As a result, very few individuals afford the benefit of learning from experience (and/or mistakes), in leading such a project or be a part of existing training projects. Especially when such experiences sometimes require a 100k++ training bill.

What further this scarcity, is how the vast majority of them will be locked up in major tech companies, and/or China.

As such. When the opportunity arises to invest and anchor them to be champions in Singapore. To build and grow AI practitioners around them - As part of our larger AI strategy. We should do so.

Notable example: Yann LeCun who leads the Facebook AI team, as one of Canada's Anchor and Champion to the pool of ~1,500 AI Creators. Alternatively, Mistral is the anchor for France.

Considering the rarity, it is easy for VC’s and Government officials to fall into the thinking of “there will be another opportunity or later”
Or defer the opportunity out of concerns regarding “unclear AI future”
And never get such an opportunity again for years.

The thing is, in a world where no one can be certain what will happen next in the field. The winners would be among those who play an active part in building this future, figuring out the answer along the way.
Before the rest of the world understand, or take action, when it is might be too late.

Like athletes, these are teams you invest in, to take a shot of leading the world stage. Without ever knowing the certainty of success of any one team, until the race is finished.

The champions, to inspire the rest of your industry / athletes

2) We do not need a multi-billion-dollar GPU cluster

It’s more a nice to have than a need.

Broadly speaking, AI models can be viewed across the following stages

• Pre-Training

• Alignment / Fine-tuning

• Deployment

In only the later 2 stages, is it more desirable for it to be done “locally”, and broadly speaking we would need “just enough GPUs”.

Pre-Training

is the only phase that requires thousands of top-of-the-line GPUs, with dedicated power and networking, to be all centralized in a single location.

This phase is highly constraint by energy cost, and typically uses largely public data for the training process. As such, it is location-independent and can be performed remotely, anywhere on this earth.

This, as a result, biased foundation model training, towards cheaper low-cost energy or cooling.

Because Singapore is not naturally cooled like Greenland or Finland. Nor do we have a large quantity of cheap electrical energy. There is little to no reason why Pre-Training “must” be done in Singapore itself.
(eg. I use Finland servers significantly, but never been once to the country)

Alignment / Fine-tuning

The next phase is when Model practitioners, or AI Engineers, take existing models, trained on trillions of parameters. Fine-tune and align such models towards a specific goal or task. With a small cluster of GPUs across thousands to a few million tokens.

During this phase, it may be common for more sensitive/private data to be included. For example, a hospital fine-tuning a medical model may be using private datasets that are not allowed to leave the nation.

So, while there is still a need for high-end GPU, with costly GPU interlink at this phase, it is significantly smaller (and cheaper) than the pre-training phase.

Deployment

After all the training, fine-tuning, and alignment. The next step is deployment. For multiple reasons, be it the latency of the request, or the legal sensitivity of personal information being used in the request.

There is a desire to bring the deployment within the same nation.

Typically at this phase, we avoid costly interlinks unless strictly needed by the largest of models, and for smaller models, may even use smaller more cost-effective GPUs instead.

3) We do not need a large team to do XYZ

One of the big trends in AI is how AI project is defined more by individuals and smaller teams, than large organizational teams.

Because a large % of their budget is spent more on computing, GPU, and supporting resources. AI-first teams tend to be substantially smaller than your typical development team.

Notable examples of this are Reeka and DeepSeek, with less than 50 people. Created foundation models from scratch, comparable to GPT 4. Midjourney is less than 20, The foundation model team for RWKV (which I co-lead) is less than 10.

It matters more what the individual contributors do, test, and decide on for an AI project, in a way that can be scaled across multiple GPUs. Then a large organization of individuals collectively.

Outside of the AI model creation space, and into the practitioner space, we have started seeing the rise of AI Practitioners, who effectively create entire million-dollar companies, by themselves, or with teams smaller than 10.

Putting the above 3 scales into context

Realistically, to build a 15,000-strong pool of AI Creators and Practitioners. We are looking at most some combination of the following…

• 3-10 AI Creator teams, of 10-100 individuals (< 1,000 total)

• 30-100 AI Practitioner Tooling teams, of 5-100 individuals (< 5,000 total)

• 200-500 AI Practitioner teams, of 1-20 Individuals (~12,500 total)

Additionally, Our existing national GPU cluster plans are probably enough when fully utilized. We can subsequently let the private market fill in the gaps when needed.

With that out of the way, let’s define the type of AI hubs out there.

The 4 major types of hubs

With examples provided with each.

• Canada: AI education and research hub

• India / Taiwan / Guangzhou: AI Enterprise Solutions & Implementation outsource hub

• Singapore Default role: A sales & management hub for South East Asia for big tech

• San Francisco & France: A tech hub for new AI companies and startup

Strictly speaking, none of these options are mutually exclusive. But it does help guide the vision of what we would want to achieve as a nation.

Additionally, while we definitely can take a balanced bet on all 4 routes. If we want to succeed beyond our default role, we need to make sure we put in the required resources for other bets to succeed

so that we do not try to do everything, and achieve nothing.

Canada: AI Education and Research Hub

Canada is currently well regarded in the AI space as one of the main AI research hubs.

All the major companies, like Google and Facebook, have set up their AI research lab with the universities. And pooled in close to $2 billion dollars, worth of investment into the area.

All so that they can have access to the growing pool of ~1,500 researchers and scientists, along with early access to the larger pool of 11,000 AI and data science students.

What is unique behind their success is how it was primarily about the concentration of talent.

The major companies were setting up operations, not because of the economic demand for AI in Canada, nor because of the startup ecosystem.

Just talent. Driven by prominent researchers at the top of their field, at Google, Facebook, etc. And proximity to the US timezone (and English!)

You can read more here on Canada investment and AI hub: https://www.investquebec.com/documents/int/publications/Brochure_IntelligenceArtificielle_web_AN.pdf

Challenges in replicating this hub today:

Pre ChatGPT, the bulk of AI research was done in universities, or jointly between commercial and public groups. It was common for the head of these research labs, to play an active role in the universities. Publishing papers jointly in the process.

Canada positioned itself as the ideal place for these researchers. By having one of the largest pools of AI students, which these companies potentially can recruit from. On the flip side, the students interested in AI research, came to the Canadian universities, as this was where the prominent head of research in multiple labs, was giving lessons from.

Post ChatGPT, various of the largest frontier AI groups, in particular the closed source ones, now avoid publishing papers, and avoid working with universities. Making it particularly hard to bootstrap this very same process from scratch.

But possibly doable, if we focus on collaboration with multiple hubs, as a means to bootstrap our education hub in AI.

Also, notice how it’s only ~1,500 researchers. And it’s considered one of the biggest hubs out there.

India / Taiwan / Guangzhou: AI Enterprise solution and implementation outsource hub

It’s no secret enterprise is spending lots of money on custom AI solutions. And are turning to their traditional solutions providers for answers in implementing their AI solutions.

To the tune of over $2.6 billion a year, for just Accenture alone. Who only represents a fraction of the overall landscape of hundreds of IT solution vendors.

So while India / Taiwan / Guangzhou, may have a limited pool of AI model creators. They have since rapidly up-skilled their pool of Software Engineers to AI engineers and practitioners, to serve the growing demand of enterprises for AI solutions.

For India in particular, this is bolstered by the recent domestic national focus on Sovereign AI solutions, due to their own national language barrier, and the desire to move away from both US and China-based solutions, due to geo-political reasons.

Challenges in replicating this hub today:

This is the culture of large enterprises, which unless we can change even within our government and large institutes, we are doomed to repeat again.

Collectively, India / Taiwan / Guangzhou currently holds their position as the world’s Enterprise Solution workforce hub, due to their large pool of technical workforce, at affordable prices.

The current demand for their AI practitioners has been an organic shift from Software Engineers, as demanded by their solutions providers.

Besides India, another growing segment lately has been Taiwan / Guangzhou. Multiple enterprises within Singapore outsourcing their development workforce to Taiwan / Guangzhou.

All of this is largely driven, by how enterprises tend’s to generalize headcount as interchangeable. As such salary price has been the number one driver of this movement - which will not be working to our advantage.

I do not have a clear answer to this problem. Beyond “somehow, magically” changing large organization mindsets.

And this is being said as someone who has worked as a vendor to 90+% of all the banks and financial institutes in Singapore and has been part of the enterprise system.

The cultural thinking of developers and engineers being equivalent or a cost center in the organization - instead of talent and asset - is a complex problem in itself that runs on a global scale.

Singapore Default Role: Sales & Management hub for South East Asia for big tech

Because of India, Taiwan, Guangzhou, and economic salary advantages. They ended up as the outsourced development hub.

Instead of competing head-on, what ended up in the software development scenes, was more enterprises leveraging Singapore as a Management hub to bridge the gaps between the software developers overseas to their customers in the region.

So while there are still developer talents hired locally, they are pushed to either manage their overseas team, or work on more sensitive, or contextually local work. Leaving the bulk of the work to their overseas outsourced development hub.

And perhaps this is alright? If this is our goal.

It’s what helps entrench our positioning as the hub for South East Asia and helps strengthen our finance sector, as all the capital flow for these operations would be flowing in and out of Singapore.

Challenges in replicating this hub today:

There is not much to be added, this is the default for us, as long as we do not be “complacent” on it. And continue our investments down this path.

Due to our strong financial services, and legal framework. This has already been our default role for software development for big tech in the Asia region.

However, this route is not going to help us create a hub of 15,000 AI Practitioners and Creators. It will more likely instead create a hub of 1,000 AI Project Managers - 2,500 AI Practitioners locally - and 15,000 Practitioners overseas.

While it will help continue to grow our finance services, most of the value and benefits to the economy will be routed overseas. While substantial, it will not be the new “Engine of Growth” we need.

One is a low-risk, low-effort, safe bet, that will give us modest economic benefits, which will certainly happen. The alternatives are huge expensive bets, that may be a new engine of growth if done right. But a waste of capital if failed.

Luckily for us, even if we make big bets, this default will still happen. Or may even be bolstered.

San Francisco & France: A tech hub for new AI companies & startup

We have seen the rapid multi-billion dollar investments and the rise of multiple startups in both France and San Francisco. Both big and small.

Every nation wants to have such a hub that has been built. In their local country and economy.

As it brings in the investors, talent, and the people.

And it is, at a mind-boggling scale. San Francisco companies receive 50+% of all global funding in AI, or US$ 27 Billion+, with France and China following behind.

And despite San Francisco sky sky-high salary costs, the reason why talent is retained and attracted is due to the ecosystem that has been built.

The reason why these startups and new AI tech companies do not overly outsource like Enterprises is because the individual contributor matters more than the head count. Where the individual salary cost matters less than the GPU costs.

I will be covering more about San Francisco and France in a follow-up article. So as such, I will leave it as it is. That it is currently the AI hub the world wants to be.

Challenges in replicating this hub today:

Capital. Every country wants a San Francisco or France AI hub.

But they do not want to put up the >$1 Billion dollar investment required to make it happen.

Secondary Obstacle - Market & Talent.

But here is the crazy thing:
Singapore literally have all the infinity stones to make this happen - if we can put it all together, and “Snap” it into place.

Deciding between an Education hub, or a startup tech hub

Because we will become the sales and management hub for South East Asia by default.

And the fact is, we will unlikely win the outsourcing salary competition for enterprises. Due to many external economic factors.

Because of that, our options are to go beyond, if we want to build that 15,000-strong workshop. Is to either be the Education Hub, the AI startup hub, or both.

So my fundamental question is….

If it’s an AI education hub:
I am not a professor at a university. I cannot advise how to build Singapore into an Education hub. So I will be skipping that. (Maybe ask the professors in Canada)

But if it is an AI Startup hub:
Because I am a Founder, an Entrepreneur, and an AI Researcher. I can help answer that question next.

Which in part 2 [here], I will line up a high-level blueprint on how we can build an AI startup hub in Singapore, and bring it up as a leader on the World Stage.

Signed Eugene,
From San Francisco

A wave of AI foundation model companies, with half a billion dollars raised - reaching their end

Throughout 2023, we see headline after headline. Of various all-star teams, pulling researchers from the top of FAANG, Ivy leagues, Open AI. To create their Avengers of AI.

Since 2024, we have seen the fall, one after another, after asking 100’s of millions after 100’s of millions. Reach their end, as they run out of money

Everyone was trying to follow the same blueprint, of openAI oversimplified

• Step 1: Get the best researchers, Spend $XX million a month, researching and building the best model out there

• Step 2: Keep researching, get results, scale up, Ask for more money for each milestone

• Step 3: ???

• Step 4: Profit; and make a billion-dollar business

The AI market is no longer the same as it was pre-OpenAI - the formula was never reliably repeatable in the first place.

AI does not remove the fundamental principle of businesses needing to make money

But, but … we funded the team that had all the brightest AI researchers, from Google, openAI, Ivy leagues …

That's exactly part of the problem, of how VC’s oversimplified the whole thing. As much as I love their work, what do you think researchers will do with funding…. Research!

Do you not know what the life cycle for a researcher is?

• Step 1: Propose a research idea, get funding grant approval

• Step 2: Get results, Write papers, and Scale up. Ask for more money for each milestone

• Step 3: Repeat steps 1 & 2, with more ideas, or bigger ideas

• Step 4: Accumulate enough paper citations, get university tenure, retire

Noticed how in the researcher's life cycle, at no point was it about monetizing, making profits, or making a billion-dollar business. If anything, research by its nature is very capital-inefficient.

While they are undoubtedly great brilliant minds in research (and I love them for their contributions). They are not the right people to turn research into commercial products. And inversely, ask for more and more money.

While leaving the commercialization process to “someone else”.

VC’s correlated PhDs & papers to commercial success. Over-indexing to OpenAI's outward appearance of success. When literally, some of the most profitable, and capital-efficient teams in AI have the inverse correlation

Wrong Conclusion: OSS / Foundation AI is too expensive?

The continuous repeat of the extremely large funding cycle of 2023, with larger and larger funding cycles, from foundation model companies. Have lead to extremely wrong conclusions from the VC markets

• That foundation model companies require half a billion dollars and more to succeed

• Open-source AI companies can't succeed.

This could not be further from the truth. GPT-4 is estimated to have a training cost of $100 Million in GPU time. And with advancements in AI techniques and technology, this cost has been going down (not up).

Add in labor, and various costs - Any company that raised beyond $250 Million and failed to catch up with GPT-4 is just being “capital inefficient”. A number that is well below many foundation model 2023 capital raise.

So why do they keep failing? It’s back to the basics of “lack of focus” and “financial discipline”. And distracted by research

This can be best reflected by, a failure of the following basic test ….

Litmus Test: “Time to pricing, and payment”

In the same naive mistake, that countless Open Source Companies have encountered, foundation model companies went on the same blind trend of “figuring out how to make money too late”.

And I say this, not only as an observer but as a passionate User as well …

I love you all, but ….

In Rancher's case: I have given talks, workshops, and evangelized ranchers into other startups. But have contributed $0.

• I use their product daily (self-hosted), and would pay to have a good managed SaaS solution, which never existed.

• The only current way to pay them is to be a giant enterprise with a crazy expensive custom enterprise contract, which I am not.

In Stability AI case: I love you, folks. As a GPU donor recipient for the RWKV OSS projects (thanks!). I was literally trying to help pay you, in some shape and form, to help improve your numbers for your investors. But gosh do you make it impossible! I had to fight your system to give you money.

• The paid image generation platform (dreamstudio), is nowhere to be found on your home page and is constantly on page 3 of Google for text-to-image. Refuses a subscription, and is only credits-based. I have to bookmark it, so I can use it every now and then when I need an AI image generated. To effectively give it a few dollars every now and then.

• The model membership, for $20/month, requires me to fill up a form that requires manual review - only to have the reviewer inform me that I do not qualify. As I was not having enough users & making enough money. In both cases I know this, I just want to give you $20/month

For Inflection AI case: How does anyone even pay you, there is no enterprise sales form, nor cloud sales process on your page.

There are already enough problems getting pricing right, for any SaaS/platform growth …

But to make it impossible for customers who want to give you money, to give you any. Is in my opinion the complete opposite of “financial discipline”, which is so self-damaging, it’s investor harm.

Unless your plan was a pure acquisition exit. If you want to make a stable company, constantly releasing wonderful open source code or AI models to the public. You need to have an income stream. Employees need to be paid, Servers / GPUs need to be paid. Investors need to be paid an ROI.

As a business, your number one primary goal is to make Money. We can discuss common good, altruism, etc - AFTER - we make sure your business, is stable enough to stay around and pay the bills while making money in excess.

Even charities and non-profits need a plan on how to get donation money. So does your business, noble goals, and idealistic open source visions aside.

This is a problem not just for AI, but any large-scale open-source project in general. A problem we have seen repeated many times. With idealistic views.

We can do better ….

On the other spectrum, is established companies with Open Source/Core models: MongoDB and Elastic. WIth Open Source/Core products that I have gotten either my startup or other startups I advise to be paying for.

The time to pricing, and payment is relatively straightforward within a few clicks.

It's so straightforward, that this revenue funnel, from one-click users to enterprises - makes them a revenue of $1.5 Billion / $1.1 Billion dollars per year respectively, with an R&D budget of over $300M per year for both.

Enough revenue, or R&D budget, to be training GPT-4 multiple times a year. Enough R&D spend, to outspend the 3 other companies I highlighted, that failed this basic litmus test.

This is despite, having everything about their Open stack being potentially an investor nightmare

• Their software stack has been cloned by every major cloud provider and competitor

• Their open source / open core offering is complete enough, that yes - I can actually self-host everything if I want to - and I do so for my own personal cluster usage

But why do I and several others, then decide to pay them, instead of their cloned offerings - to a tune of billions of dollars

• Production Reliability: Scaling up AI and Databases safely and reliably is incredibly hard, especially in production where you cannot afford any downtime.

• Convenience in scaling up and down: Related to production reliability

• Convenience from the Source: Ease of upgrade and access to updates, before any other providers

• Reasonable Pricing: Without crazy per-seat pricing, it's not much cheaper to get the same performance/service offering on your own, once you factor in upgrade processes required, hardware redundancy, etc. Both Elastic, and MongoDB inversely, get to leverage their economics at scale, and benefits as part of their margins.

• Multi-cloud offering: allow me to easily move between AWS, GCP, and Azure if needed

• Enterprise compliance and security (ISO, and SOC certification processes)

Essentially make it more attractive for me to use your solution and hardware. Then my own cloud hardware, and we the user will pay.

Wrong Conclusion: Is the pricing page all I need?

It is a start and is infinitely better than being user-hostile in making money.

It may be sufficient to get to the thousands, or maybe millions in revenue. But if you want to reach billions, it's all about the details …

Moving from research to production, and sales is a muscle!

A muscle that rarely any one company gets right the first time round, a muscle that needs to be constantly trained and iterated.

Notice the list of reasons previously, on why I and many others choose to pay for the commercial offering. Despite even having the expertise to self-host on our own?

Without exercising the muscles from converting a research product into production and sales, it is easy to make the following mistakes

• Poor production reliability: Rate limits, Downtime, all constantly puts off users

• Missing features that the customer wants: from paid offerings, which the open source offerings provide

• Poor pricing: Users are not idiots, we will shop around, and get pricing drastically wrong compared to your peers. And we will switch. Inversely, price too low, and the company gets burnt.

• Missing conveniences: A better UX/DX, is a very large reason why your commercial offering will be preferred over DIY.

• Bad sales processes: Your enterprise sales team, needs time to find Product Market Fit, needs time to scale.

While I can sing the praises of Mongo, elastic, or even supabase (Postgres). I can also point out various open-source technology projects, which failed on some version of these points badly enough. That their users were pushed into self-hosting or other solutions.

There is no one-size-fits-all solution to these issues, there is only going to market, and iterating with the users. Getting this right for B2B / developers and enterprises is complicated enough entire investors and courses is specializing in this segment (eg. heavybit)

We know all of these things, We have learned it through the cycles of various deep tech startups, and B2B SaaS. Sure investors may have been blinded by greed to repeat OpenAI, through oversimplified pattern matching ( Big name + AI = Profit ?? ).

But why do founders, even business founders, avoid building those muscles - when they know better ???

The problem of incentives from VC’s …

Foundation model / AI companies by their very nature are deep tech. It requires a fairly deep capital of 50M and up.

On the flip side, consumers are very demanding, and besides notable exceptions, are mostly only willing to consider paying once a model reaches GPT 3.5 class and up approximately.

1) VC’s Overfitting to the Product Market Fit revenue rocket

One of the notable issues, that founders constantly discuss - is that once you have revenue - VC’s tend to only focus on the revenue and Product Market Fit curve. Which boils down to: Is it a rocket or not?

This creates a weird situation, where strategically speaking we founders are aware of the uphill sales cycle for anything below a GPT 3.5 class model.

As such, It makes more sense to avoid any revenue until you have an AI model approaching the GPT 3.5 class, after which you can start building a platform. This process is something even several VCs will advise founders on, to maintain focus …

Knowing the tall task involved, many opted to focus on benchmark and open source adoption first before any commercialization … but that has its own weaknesses …

2) Worry about the revenue later instead? You are going to get sniped.

This is what happened to Mistral, as they scaled their model up to Mixtral …

Only to find competing providers, to not only enter the market first to the public before them. But to do so at a lower price point. With their model.

All of which was to be partially expected - because, while Mistral was busy entirely just creating the model - they never built that muscle for bringing their very own model into production and optimizing the user convenience to the sales process. They were isolated from the users.

Inversely, together AI has already been, running the Mistral 7B model. Iterating with the users, on every factor, from convenience to price. It may not have been a major success, but they have gotten the iteration process going. As such, when the mixtral model came out, they were more prepared than the mistral team to publish their API for the model.

Together AI has been exercising the production shipping, and commercial muscle. Mistral didn’t

3) Making it worse: build it close source, behind the scenes? Scale up when we are ready

Basically what Inflection, and a few others did.

• If they avoided revenue, they were forced to be working on the model, without user feedback …

• If they accepted revenue, they will face an uphill task of making consumers care … because they were neither GPT 3.5 class nor open source which they can tinker with … Additionally investors would be doubting them in the early stages due to 1)

• This is also why - nearly every AI startup is a “beta” in the early stages, to have some form of customer users and feedback loop. Without committing to the revenue hockey stick.

Basically, you escalate the stakes. You either clearly beat and provide a strong reason to move from the existing close source model with a Unique Selling Point. With very limited user feedback, till you get to that point.

And if you fail, you will have ended up creating a model, which no one wanted (as it was not adapted to user feedback). A model and platform that no one cared about (as it is not open source), with little to no revenue nor user adoption.

Sounds hard right? Well there is an answer … An alternative …

How Should it be done then? How does one build a foundation model company?

You need a founding CEO

• who has both enough domain expertise and respect in AI to lead an AI team

• with business sensibility,

  • to actually steer the company to profit

  • and not burn more than 100~200 M to make GPT4 class AI

Why not a business CEO, with a AI researcher CTO?
You will need them both to be very very aligned. Hard decisions will need to be made that involves a quarter to half of the company capital. A decision which can only be made with informed context of both the AI training process and business goals. Decisions which either side may fear making or misjudge, due to lack of context of the other side.

With a founding CEO / CTO / Team, that has some mix of the following

• dataset and AI model training and scaling experience

• who has experience building and scaling resource-hungry infrastructure platforms (for inference)

• sales and marketing muscles with either (or both)

  • a DevRel team to help promote the platform adoption

  • or an Enterprise sales and partnership team to close the deals

What if there are talents lacking in specific segments?
That is fine, its hard enough to find the combination of talent required for building and scaling AI models (There are less then 1000 individuals who has experience training a 7B and above model from scratch). Let alone everything else. Discuss with the founding team, a recruitment plan for the required gaps in the team.

With a plan to scale in steps, both the model and platform

• Stage 1: Something to prove the concept / technical approach

• Stage 2: Initial platform and model, which can start closing some customers (even if it is ineffective, unscalable, or slow)

  • This can ideally be within a specific market niche or use case

  • Helps ensure all initial scaling issues on the platform are tackled, and sorted out

• Stage 3: Scaled up, highly performant model and platform, tuned heavily to the user needs (where the hockey stick begins)

  • Ramping up on sales and customer adoption

  • Maybe split into smaller sub-stages (eg. GPT 3.5 class, GPT 4 class)

• All in under 2 years, because AI moves fast

• With Investors, who are willing to commit to such a plan, step by step.

Can’t we skip stage 1 & 2, and go straight to 3?
Unless you are able to find the required talents who have done it all before. Which you do not, as they are all concentrated within OpenAI and Anthropic (not even google counts). Across all the various key disciplines. Then maybe you can remove stage 1 & 2?

And even then, that would be a high risk move to an already high risk endeavor. Its better to just sort out the bumps, across a few months, and the customer feedback loop. To get this right.

For context, openAI had the time between 2018-2020 for stage 1 (GPT-1 & 2). Followed by 2 years+ of stage 2, with early user traction and iteration (GPT-3) between 2020-2022 And only stage 3, towards the end of 2022, early 2023.

But yea, many VC’s loved to ignore the first few stages, and only focus on replicating the last stretch. Without realizing how hard and essential it is to make sure to sort out the platform at scale. After all this is not just a “SaaS template” + SuperBase combo. The success and failure is in the details.

And dun forget to make a profit?

By selling the AI product, at a price point that - actually makes money this time!

OpenAI is reported to be making losses of near $500M a year, or over a million dollar a day. According to various reports, including buisnessinsiders.com

The last part of making profits, making money, over the cost of goods - is important because

• You can cover all bills, to be default alive

• You can funnel the revenue to future models, and customer growth, To grow future revenue

• And recursively scale up the business, and AI model in the process.

Because why not? The market is hungry, over a 100 billion dollars a year hungry, if you get the pricing and scaling right even with existing AI models (GPT 3.5 and GPT 4 class)

Sounds like a saner business plan - better than trying to raise 7 Trillion dollars right?

A self-fullfilling loop that keep scaling past even AGI?

Well, do we have something for you …

Guess what! - that's basically what, me and my team are doing here are Recursal AI are doing.

• We are built using a new AI architecture: That can make a profit, as it’s 100x++ cheaper on inference compared to existing AI architecture (transformers).

• We plan to make not just revenue, but profits: Its part of our founding story, we grew from the ashes of un-stable AI companies. And never want that to happen again

• We do not plan to raise more than $250M: Because, we have always been capital-efficient, and plan to stay that way.

• We just finished Stage 1 (while spending <$2.5M):

  • We proved our architecture, can be scaled like transformers and even outperform LLaMA 2 7B

  • We achieved SOTA status, as a 7B multi-lingual model

  • Multiple orders of magnitude more capital efficient then any LLM group

• We are now at Stage 2:

  • We have our initial commercial cloud platform, and strategic partnerships (more details to come)

  • We have initial customers, giving us a tight feedback loop, and pays us

  • We intend to find PMF with our Unique Selling Point at this stage, across either:

    • Affordable, High throughput support

    • SOTA Multi-lingual support

    • Support for enterprise private cloud / on-premise deployment support

• We are scaling to GPT 3.5 for Stage 3A, Followed by GPT 4 for Stage 3B

• All while building the open-source AI model, open source under the Linux Foundation

• And creating that looping wheel, of profits, powering model improvements, in a stable recursive manner.

  • That makes both investors and employees happily rewarded.

  • A loop I wish to see repeated across more competing open source models.

If that makes sense to you, feel free to reach out

• For investors: founders@recursal.ai

• For those who want to join us: hr@recursal.ai

Recursively: Making AI, for everyone, in every language

~ Until next time 🖖 live long and prosper

The following is an abstract of some thoughts i had from the RWKV podcast found here: https://www.latent.space/p/rwkv#details

Why Alternatives Matters ?

Transformer architectures are currently at their peak with the AI revolution of 2023. However, in the rush to adopt them due to their recent success, it's easy to ignore alternatives that we can learn from.

As engineers, we should not take a one-size-fits-all approach, using the same hammer for every nail. We should find trade-offs in every solution. Failure to do so would trap us within the limitations of one particular platform, while remaining blissfully unaware of all other alternatives. This could potentially hold back development for years.

This problem is not unique to AI. It's a historical pattern that has repeated itself since ancient times up to the modern day.

A page from the SQL war history...

A notable recent example in software development was the trend towards NoSQL when SQL servers started hitting physical limitations. Startups everywhere made the move to NoSQL for "scale" reasons, despite being nowhere near those scales.

However, over time as eventual consistency and NoSQL management overhead emerged, and hardware capabilities made giant leaps in SSD speed and capacity, we've seen a recent trend back to SQL servers for their simplicity and now sufficient scalability for 90%+ of startups.

Does this mean SQL is better than NoSQL or vice versa? No, it simply means each technology has preferred use cases with pros/cons and learning points that can cross-pollinate among similar technologies.

I hope you view this article as not an attack on transformers, but a healthy appreciation on alternatives. With that, lets get on to the main topic

What is the biggest pain point for existing transformer architecture?

Generally this covers compute, context length, datasets and alignment. For this discussion, we'll focus on compute and context length:

• Quadratic computational costs due to O(N^2) increase per token used/generated. This makes context sizes >100k incredibly expensive. Which impacts inference and training.

• Current GPU shortage exacerbates this issue.

• Context size limits attention, heavily restricting "smart agent" use cases (like smol-dev), forcing workarounds. Larger contexts need fewer workarounds.

So how do we solve this?

Introducing RWKV - a linear transformer / modern large scale RNN

The following summarises the RWKV paper at a high level: here
For a deeper technical dive on RWKV architecture, see the: wiki here

RWKV, along with Microsoft RetNet, are among the first of a new class called "Linear Transformers".

It tackles the above 3 limitations directly by supporting the following

• Linear computational cost regardless of context size.

• Lower requirements allow reasonable token/sec output on CPU (especially ARM) in RNN mode.

• No hard context size limit as an RNN. Any limits in docs are guidelines - you can finetune beyond them.

As we continue to scale up AI models, to large context sizes. From 100k and beyond. The quadratic computational costs starts to exponentially increase.

Linear Transformers, however instead of abandoning the Recurrent Neural Network architecture and their bottlenecks which forced them to be replaced by transformers.

Took lessons learnt from making transformers scalable, and re-engineered RNN's to work similar to transformers, and remove all those bottleneck.

Bringing them back into the race with transformers, in terms of training speeds. Allowing them to both operate efficiently at O(N) cost, while scaling past a billion parameters in training. All while maintaining similar performance levels.

At 14B parameters, RWKV is the largest open-source linear transformer, on par with GPT-NeoX and others on similar datasets (e.g. The Pile).

But what does this mean, in simpler terms?

Pros

• 10x++ cheaper inference/training than transformers at large context sizes

• Can run slowly on very limited hardware in RNN mode

• Similar performance to transformers on same datasets

• No technical context size limit as RNN (infinite context!)

Cons

• Sliding window problem, with lossy memory past a certain point

• Not proven to scale beyond 14B parameters yet

• Less optimised and adopted than transformers

So while RWKV isn't yet at 60B+ parameter scales like LLaMA2, with the right support/resources it has potential to get there at lower cost and larger contexts. Especially as expert models trend towards smaller and efficient.

Consider it if efficiency matters for your use case. But it's not the final solution - healthy alternatives are still key.

Also one last thing i forget to mentioned:
Linear Transformers typically do not use classic Transformer attention, but an approximate equivalent, meaning ...

Attention is Not All You Need

Other alternatives and its benefits we should probably learn from

• Diffusion Models: Slow to train for text, but extremely resilient to multi-epoch training. Figuring out why could help mitigate the token crisis

• Generative Adversarial Networks / Agents: techniques could be used to train desired skillsets, to a specific goal without datasets. Even for text based models

Disclosure: I am a code contributor to the RWKV project, and an active member in the community.

However it does not mean I view RWKV as the final answer, even if in the future where "Linear Transformers" beat out traditional "Transformer" networks in the future, research into alternatives is still important for healthy ecosystem development.

I strongly believe there is no such thing as "one architecture to rule them all"

Additionally, i am aware there has since been developments like TransformerXL to more efficiently scale transformers, somewhere between quadratic cost, and linear cost. However with very limited known implementation of this method in public, it is hard for me to evaluate its pros and cons.

While GPT 3+ or ChatGPT, prompt engineering is easier to understand intuitively. With many guides and examples available on the web, and social media. Such as

• https://help.openai.com/en/articles/6654000-best-practices-for-prompt-engineering-with-openai-api

• https://github.com/f/awesome-chatgpt-prompts

Embeddings, requires programming, and are less understood, due to various counter intuitive behaviour on how it works. But, is an extremely powerful tool for search, or to be used together with existing text-based models, for various other possible use cases.

Embeddings is arguably an equally powerful tool, within the AI toolkit, for instruction models. Due to its ability to handle searches across different words and sentences, or even entire languages. Focusing on searching for the relevant document, for any query.

For example, it can be used to power searching and answer from an English-based documentation. In English ...

Or Japanese ...

Or any other languages that the AI model supports. Embeddings can also be used for other tasks such as question-answering, text classification, and generating text.

Note that this article focuses on the search aspect, the answering process is in a follow-up article.

What is an embedding vector?

To generate an embedding vector, one will use an embedding AI model, which converts any text (a large document, a sentence, or even a word), into an "N dimension array", called a vector.

For example a sentence like How do I write a UI test script with Uilicious?

Can be converted into the following array (called a vector) via OpenAI text-embedding-ada-002 model: [0.010046141, -0.009800113, 0.014761676, -0.022538893, ... an a 1000+ numbers]

This vector represents the AI model's summarized understanding of the text. This can be thought of as an "N-word summary" written in a language that only the AI can understand.

Where related documents will have a close distance from each other, based on the AI's understanding of the document (and not its text). This is a major leap beyond the simple keyword searches of classic search engines, as it can handle variations in sentence structure and language (provided the AI model is trained to understand said language).

Take the following as a hypothetical example, oversimplified inaccurately into 2 dimension space so that it is easier to understand:

Which can be visually presented as the following in 2D space.

For example, D1,2,3 are all documents related to how to use Uilicious in various ways and are grouped together in one cluster

D4 and D5, being simply links and having no inherent value beyond that, are grouped separately in another cluster.

Furthermore, D1 & D2 are further grouped together, as they are about Uilicious testing commands, using our very own JavaScript-based testing language.

While D3 is grouped separately, as it is regarding using the web driver protocol on our infrastructure directly, which is intended for a different use case and audiences.

Similarly, for Q1 and Q2, despite the drastic differences in sentence structure and language, because it is essentially the same question, the two questions are grouped together. Additionally, while the question could technically be interpreted both ways (using Uilicious test script, or webdriver protocol), because the question implies the usage of Uilicious test scripting over webdriver, its location is "closer" to D1 & D2, and further away from D3.

As such, despite having huge overlaps in keywords, these nuances in groupings are captured by the AI encoded within the embeddings.

In reality, however, instead of an oversimplified 2 dimension array which is easy for humans to understand, an embedding can easily be a 1,000+ dimension array. This array is unique to the specific AI model used, and cannot be mixed with another AI model's embeddings.

Math notes: N-dimension math is not compatible with 2/3D math

While the oversimplified 2 dimension examples, are good for understanding the high-level concept of grouping relatively to one question (or one point of view), it does not accurately represent an N-sphere, which is the mathematical construct used to describe a space of N dimensions.

Due to complicated N dimension math, you can have situations, where A can be close to B, B can be close to C, but A and C can be considered far from each other. Which is an extremely counter-intuitive gotcha.

This is known as the Curse of Dimensionality

Such distances are only useful when used relative to the same point, and the formulas used. Which can be calculated either using

• Euclidean distance: also known as Pythagorean Theorem on steroids, it is the most common distance metric used and is the straight line distance between two points in N-sphere.

• Cosine Similarity: it is a measure of angular distance between two points in an N-sphere, and is useful for measuring the similarity of documents or other vectors.

• Manhattan or Hamming Distance: these two metrics are used to measure the differences between two vectors, and are useful for measuring the "edit distance" between two strings.

While the effectiveness of each formula has their respective pros and cons, for different use cases. For textual search, it is generally accepted that Euclidean distance "works better" in most cases, and "good enough" for cases where other methods edge out.

All of which, is used in effect to reduce N-Dimensions, to a single dimension (distance), relative to a single point. This as a result, would mean groupings can / may change drastically depending on the question asked.

This "relativity" nature of distances, makes classic database search indexes ineffective.

If that makes no sense, this is how 4 dimension space is visualized properly with math N-Spheres.

Now imagine 1,000 dimensions? Yea it makes no sense.

So without derailing this topic with a PHD paper, I will summarise this as just trusting the math professors.

All we need to understand is that in general, the closer the distance is between two vector embedding points, the increased likelihood they are relevant to each other.

From a practical implementation standpoint. Start with using Euclidean distance first. Before considering using the other formulas which are fine-tuned for better results through trial and error for your use case (not recommended).

Searching embeddings with a vector database

So given that we can convert various documents into embeddings, we can now store in within a database and perform a search with it.

However, unlike an SQL database searching with text, both the search and the data being searched are the vector embedding itself. This means that traditional database search indexes are ineffective when it comes to searching for embeddings.

We can have all your documents embeddings, precomputed, and stored into a vector search database. Which can then be used to provide a list of matches, ranked by the closest distance.

This can be done using existing vector databases such as

• REDIS: a popular open-source database that can be used to store vector embeddings and search them efficiently.

• Annoy: a library created by Spotify which uses an optimized algorithm to search embeddings quickly.

• FAISS: a library created by Facebook which provides efficient search algorithms for large datasets.

One major thing to note, vector search "database tech" is relatively new. Where a large majority of the vector search databases were designed for use cases found in companies like Facebook, Spotify or Google, with record sets in the size of millions or billions. And may not be optimised for small datasets.

This is going to be a constantly changing field in the next few years, here is a github 'awesome-list' to help track and find future vector search databases

As such, in general, we found for small datasets of (<10,000 ~ 100,000 embeddings), keeping the embedding dataset in memory and brute-forcing euclidean-distance-squared is "good enough" for many use cases, and will sometime outperform formal database solutions (which will have disk/network overheads) with something like the following.

The obvious downside to this approach is that the entire dataset must be small enough to fit into memory without the overhead.

Regardless if you are using local in-memory embedding search, or a formal vector search database.

That's it!

Embedding search is just a sort and rank algorithm that works flexibly with various languages, and scenarios. The question for you as the reader is how you can use it. It can be used as it is as a possible google search replacement, or together with other tools, from chat, to games. The possibilities are endless and open to exploration.

~ Until next time 🖖 live long and prosper
Eugene Cheah @ tech-talk-cto.com

All Images used, with their proper attribution

• Man in magnifying glass by Marten Newhall on Unsplash

• Zach Galifianakis Math meme

• File drawers by Jan Antonin Kolar on Unsplash

• Dream big wallpaper by Randy Tarampi on Unsplash

If we don't sort out the legal issues surrounding AI-generated content properly, it could have serious implications for the future of thousands of jobs.

As such, I believe, It's better to address these issues now rather than later, as the potential damage grows with the increased use of AI across all industries.

Throughout this article, you will see the term "works" frequently used to refer to any content to which copyright can be assigned, including images, artworks, and written text. To avoid confusion, the user of the AI model will be referred to as the "human operator", this is separate from the “creator” of the AI model itself.

For the purposes of this article, we will primarily be discussing the legal issues surrounding AI art generators, as they are currently at the forefront of this debate. However, the proposed solution discussed here is applicable to all AI models, including text-based ones.

A crash course on the AI copyright debate, and a disclaimer

To oversimplify an extremely complicated topic that could fill multiple legal essays on its own, the current legal status of AI-generated content is undecided, with lawyers and members of the public advocating for either side.

One of the high-profile cases (which is not finalized) is how the "US Copyright Office" has been going back and forth on the copyright status of an AI-generated comic (link here)

At its core, the AI copyright debate revolves around three major questions:

• Is it permissible (possibly under the concept of fair use) to use copyrighted materials to train an AI?

• Can AI-generated works be copyrighted?

• If so, can an AI model be co-assigned copyright of a generated work (e.g. an artwork) along with its "human operator"?

In general, these are the major arguments for AI.

• Proposition 1: AI learns and creates (words, music, art, etc) like a human

• Proposition 2: AI is just a tool

With the following 3 major opposition

• Opposition 1: Only "real" humans are eligible for copyright

• Opposition 2: Fair use is not automatic and is evaluated on a case-by-case basis

• Opposition 3: Lack of consent, from data sources used in the dataset

There are many additional arguments for and against AI that have been made by both sides, but they are not included here because they are either not as significant as the key points or they are based on ethical considerations rather than existing legal codes.

For example, one notable argument that was left out: is the "for progress" proposition. On how AI should be allowed to learn from copyright materials to encourage progress in society, and that resisting such progress is harmful to society. This is an argument based on ethical and societal considerations and not an existing legal code.

As I am skimming the details of the debate, if you want a more in-depth read, you can consider theverge article - The scary truth about AI copyright is nobody knows what will happen next

Finally, to state the obvious, I am not your lawyer, and the following is not legal advice.

Also, copyright laws differ drastically from country to country (Japan, for example, does not fully recognize fair use).

Now with all of that, out of the way ...

A Proposal to Classify AI-Generated Content and AI Models as Derivative Works with Attribution*

*with various limitations depending on the process used in each step

More specifically, the following sequence is proposed:

• Dataset as a collection of work, from its individual data sources*

• AI Model as derivative work, from the dataset, together with the AI model creator

  • If an AI model is used to train another AI model (instead of a dataset), it is classified as a derivative of the base model, its training data, and its trainer model.

• AI Output as derivative work, from the AI model, together with the "human operator" who provided a "prompt/instruction"*

This proposal allows us to divide the copyright problem into distinct steps and rely on existing legal frameworks and case laws, particularly the legal framework revolving around "Collaborative authorship" of open-source code. It also helps to answer the three questions distinctively:

In this framework, it would also allow answering the 3 questions distinctively.

• Is it permissible, under fair use, to use copyrighted materials to train an AI?

  • No, by default, unless it allows derivative work*

• Can generated works, be copyrighted?

  • Yes*

• If so, can an AI model be co-assigned copyright of a generated work (ie. artworks), together with its human author (user of the AI)?

  • Kind of. It would be attributed to*

This proposal avoids being stuck in the debate of whether an AI performs the creation of work "like a living being."

It also helps to create a clear path for all parties involved in a fair way:

• For artist

  • It gives them control over their own unique brand name, and art style, allowing them to commercialize it where possible

  • It does not stop AI from being used to generate artwork in known older art styles

• For AI Model Creators

  • It provides a legal framework for them to create and commercialize AI models safely

• For Human Operators

  • It allows them to generate AI artwork

  • It allows them to potentially commercialize such artworks for their own personal or commercial uses.

Understanding permissive license (usually with attribution clause)

One major factor that is often overlooked in the debate is how copyright is not binary, and how permissive licenses lie between full restrictive copyright and no copyright.

This proposal intends to take into account how permissive licenses are treated in the process and intends to use such licenses as a means to navigate the legal debate.

Because permissive licenses have been a hallmark in the open-source copyright movement for software coding (e.g. MIT license), this will help lay the groundwork for the proposed approach to existing case laws. (The list of open-source licenses can be found here).

Outside of the software industry, the Creative Commons is its most prominent variant, with over 2 billion works tracked in circulation.

Broadly speaking, permissive licenses are used within three major categories:

1. Content that cannot be used: Share Alike, and No-Derivatives

• No Derivatives is as obvious as it gets, No-Derivatives

  • Creative Commons BY-ND and BY-NC-ND

• Share Alike, or CopyLeft licenses, require the existing license to be copied over into the license of the derivative work. While technically this is possible within smaller focused datasets, for practical purposes this should be avoided due to the logical challenge when merging multiple "CopyLeft" licenses.

  • Creative Commons BY-SA and BY-NC-SA , for software code, an example would be the GPL/AGPL/LGPL licenses.

It is worth reminding the tech bros that no consent (to make derivatives) means no consent.

2. Content that can be used: Typically with attribution

• Derivative and distribution are allowed with attribution

  • Creative Commons BY, for software, an example would be the MIT license

• Derivative and distribution are allowed even without attribution

  • Works in Public Domain, or licensed under Creative Commons Zero

3. Content that can be used: For non-commercial use cases

• Distribution and derivative with attribution, limited to non-commercial use.

  • Creative Commons BY-NC

Placing an early prediction, there will be a derivative of the Creative Commons license which specifically prohibits the usage with a dataset or AI training.

Maybe BY-NADD (No AI nor dataset derivatives)

Important Note: Limitations MUST be carried forward

Permissive licenses are typically written in a way that enforces derivative works, and any collections including their work, or its distribution, to have the same limitation (attribution, non-commercial requirements).

Measures must be taken to ensure the same limitations apply to the dataset, AI model, and their generated content.

Failure to do so would break the current proposal, due to potential loopholes that may allow commercial companies to fund the creation of public datasets under "fair use" only to eventually use them commercially.

For example:

• LAION.AI - was commissioned to build and open-source a "Non-Commercial" dataset

• Stability.ai - built an AI model, based on the LAION.AI dataset, without the "Non-Commercial" restriction, and released it to the public

• Dreambooth.ai, and countless other commercial AI art generators - used the model to build their paid commercial AI art generator.

The end result is, despite starting out in a way that "does not create a competing product," due to its open-source nature, it ends up as a tool, that can be used in commercial settings to replace specific living artists identified by their name, who had their images scrapped within the dataset without consent.

How does it work in practice? - when making AI models

By classifying, downstream datasets, and AI models, as derivative works of their upstream content, copyright can be respected across a chain.

This is similar to how open-source projects work in the software industry, allowing existing laws to work on each layer.

• Dataset layer: Will be based on existing web-scrapping laws

• AI Model layer: Because the final AI Model can be treated as "Code" (it literally can be exported as code), existing open-source laws can handle it. A clear legal path will clear the way for commercial adoption

• AI Output layer: If we treat the AI model as code, with licensing restrictions that force attribution to its output, it falls into place with several existing code licensing models.

In all cases, attribution needs to be maintained up the chain.

For example, all AI models must attribute their datasets, which must in turn provide attribution for their sources.

Once this is established, there will be three major types of AI models:

• Commercial use-friendly AI models, which can be used in commercial settings

  • Creative Commons `BY` or something similar

• Non-Commercial use AI models, which inherited the various "No commercial use" restrictions from the original data

  • Creative Commons `BY-NC` or something similar

• Privately licensed AI models, which will typically be based on a "Commercial use friendly AI model" and finetuned for specific licensed use cases

For right holders (ie. artists), who do not want any of their works to be used to train an AI model, or to be used by others as derivative work. They decide and stay in control, via the licensing terms.

For rights holders, who do allow derivative works, they get to decide if it's restricted to non-commercial uses (non-competing).

Finally, right holders, especially those with very unique styles, can sub-license their data, or AI model, and set their own terms of use. Typically as a way to commercialize their unique style.

How does it work in practice? - when using AI models

Using the AI model is more nuanced. Depending on if the AI is adding IP value, or not, there are two major categories that need to be outlined.

AI as an Intellectual Property generator

The general rule of thumb is that the output would be the derivative of all three participants: the AI model, the human operator (via a prompt typically), and the input material (if any, for example, editing or redrawing an existing image).

This is because the final output would literally be a derivative of all three inputs.

If any of the three enforces a copyright clause (ie. attribution, no commercial use), so will the derivative output. If the AI model follows the Creative Commons model, and the "Human Operator" has full rights to the input, the "Human Operator" would have full rights to the output, with the attribution clause limitation (and the no-commercial clause for no-commercial use AI).

If it's just the "Human Operator" and the AI, the above structure still applies.

AI as a reduction / transformation tool

This is currently limited to the AI text model (as far as I know), where the AI is used in a way that does not add IP value. For example in "Short Summarization, Labeling, or Factual Extraction".

In this case, if the output has no additional content added and is simply the reduction of the input, or transformation (eg. translation), per the user instructions, and was done so in a way that is clearly mapped to the instruction - the AI has arguably not "added any IP or context".

As such, in this case, the AI model should be treated as if it was a tool/extension of the human operator. And the output copyright should be as if the "Human Operator" performed the said instructions manually. (Need legal feedback on this segment)

Will this stop AI from taking artist jobs?

Quite frankly, No, the bar has been raised for being an artist.

If the artist's primary job was commissioned to render artwork in a commonly known style, that is out of copyright (eg. Vincent van Gogh style) or is very common in the public domain.

However, if the artist were to make or remix a newer style, that is unique and distinctive to them.

Then Yes.

This prevents their new style or likeness, from being copied. And allows them to commercialize and be rewarded for their style on their terms. With time, like all artworks - it enters the public domain.

Ultimately, allowing them to make a name and career for themselves, for their unique style.

Into this, strange new AI world

Alright, what's next?

As I am also neither a professional artist, lawyer, nor politician. All of this may just end up being a programmer screaming into the crowd.

However, if this approach makes sense to you, especially if you are an artist, do feedback, forward it to as many legal friends, and start organizing a movement towards this.

For any lawyers who wish to refine this to a legal paper, please do so, reach out to me, and provide credit.

The only paper I can find exploring this angle is "AI Derivatives: the Application to the Derivative Work Right to Literary and Artistic Productions of AI Machines"

At the end of the day, I do wish that we all steer in a direction where, both AI and artists, can work together hand in hand. Either with what's proposed, or any other alternative proposal that respects both sides.

~ Until next time 🖖 live long and prosper 🚀
Eugene Cheah @ tech-talk-cto.com

For more on how AI works by dreaming,

Tech Talk CTO

Introducing AI / ChatGPT, and how it dreams in 2022

This article focuses more of the current state of AI, on 5th Dec 2022. As a means of briefing others outside this industry, what has been going on recently, and what are its limitations. With a focus on recent developments in chatGPT, over long term speculations…

Read more

3 years ago · 1 like · PicoCreator

All Images used, with their proper attribution

• Hand Stacked Coins With Justice Scale by focusonmore.com under CC BY 2.0

• Race Car Photo by Abed Ismail via unsplash.com

• Goals and Plans Photo by Content Pixie on Unsplash

• Creative Commons Licensing chart by kathleen_morris

• Stop Hand Photo by Nadine Shaabana on Unsplash

• Pole Vault Photo by Frans Vledder on Unsplash

• Sunrise Photo by Nicole Avagliano on Unsplash

The idea of a 10x developer, or a rockstar developer, being better than every other developer at everything, is a myth.

Instead, you should strive to be a 100x developer in very specific situations, a 10x developer in some, and a 1x developer in as many situations as possible. It's important to be aware of the areas you excel at and the areas that you may need to work on.

Unless you work for a large tech company, it's likely you'll be wearing multiple hats. On a daily basis, you'll need to be able to switch between front-end, back-end, and language-specific development, such as Java or TypeScript. This is due to the ever-changing nature of our industry and the sheer number of ways to achieve the same tasks - like printing “Hello World” in over 400 different programming languages.

It's impossible to be good at everything, but it is possible to be great at something.

To elaborate, I'd like to share a story from over a decade ago which was a life-changing moment that had a huge impact on my career.

How I Became a 100x Developer - as a Junior

I worked with a large MNC and was part of a team that was struggling to keep up with demand.

They had built a custom Java web application that thousands of employees were using daily, but as the number of records going in and out of the SQL servers scaled up. Things were slowing down. They had already upgraded the SQL server to its limits and needed new solutions. An obvious one was to use memcache to cache some of the results.

Unfortunately, memcache had been banned due to some security incidents in the past and the lack of HA. The managers had been trying to get approval for memcache for over a year but failed. Meanwhile, the devs fought an uphill battle as every 10% improvement was negated by double that in user growth.

This is where I came in, and joined.

I had no idea about their past 1 year of struggle and had been brought on board as a junior developer to help make some minor improvements to the UI, while the seniors focused on performance.

However because, I was very annoyed by the slow performance of the dev server. And I had been playing with Hazelcast - a brand new Java-based server clustering and caching library.

I used it as an alternative to memcache. And was able to get it running on the application, which had no special restrictions or special approval requirements, that killed all previous solutions.

And had a working prototype with a demo ready within the same week. For one page in the platform, the API calls went from more than 10 seconds to under a second. It was a game-changer.

The whole team jumped on board, and within a month, we had a "clustering solution which is definitely not memcache" in production.

As my team manager and his boss treated everyone to a banquet, my manager said to me, that when he brought me on board as a smart kid, he had never expected me to be a 10x or 100x programmer who could solve their problems overnight.

And that statement struck me hard.

Because I was an imposter – nothing but a lucky imposter.

The Lucky Imposter – Who Was a 0.1x Developer

Technically, I was not a 1x or even a 10x developer. I knew less than my teammates, and I was learning on the job.

My peers could do the front end better than me and the team knew how to optimize SQL (and taught me, thank you!). My biggest credit was that I was able to do the equivalent of “npm install miracle” and read its manual on how to configure it.

In hindsight, there were many issues I had dodged by luck.

• Politics (which blocked memcache in the first place)

• Use case (we had no write contention safety, meaning if 2 edits happen in the same ms, the cache will be badly updated. Thankfully, this rarely ever happens)

• Stability (v1 of hazelcast crashes like crazy, thankfully the API server was designed to be deployed in HA mode, so this was an annoyance to the infra team at worst)

• Security (Having an open port to all data on the API server in a cluster, was a bad idea on my part, and a very junior mistake until another senior thankfully read the docs on how to secure it)

• Language stack (if the server was in Python, .NET, C# or Ruby, I would have no idea how to even include a local cluster cache)

I was lucky to have a handful of one-trick ponies, which made me 100x in the right situations. Which were only possible because of the foundational work put in place by the rest of the team.

I was also lucky that my managers and bosses allowed me to choose the problems I wanted to fix, as they fast-tracked me, and threw me across multiple teams to "fix their problems". While allowing me to maximise my luck-to-impact ratio and repeat the “miracles”.

With experience, I also realised the opposite – I am slow at front-end development. This was what I was originally supposed to be doing before I was fast-tracked as a 10x engineer and it made me reflect a lot over the years. As it could have been the path I would have been stuck on.

It also struck me that in hindsight, having "mastered" hazelcast, and various other technologies. How much it would have failed for other teams, even in the same organisation, were it not for the unique situation, and safeguards, which should have been credited to the rest of the team.

I struck lucky, as a junior, if anything else.

This is why Senior Developers Tend to be 10x Junior Developers

Once we look beyond the generalisations of engineering, it becomes clear that everyone has their skills, which can range from 100x to 0.1x.

Over the years, as I moved across many teams and projects, I eventually became a “senior” developer. During this time, I realised that seniors generally have more skills, and know what they are good and bad at. They can proactively inform their managers, and prioritise tasks accordingly. This doesn’t mean they know how to do everything, but it means they know what not to do.

Juniors, on the other hand, struggle with this as they lack the experience to know what they are good at or bad at. There isn’t an easy way to find this out other than to just “try”.

Once this is understood, it becomes apparent that it’s all about maximising the number of things you know you can do reliably and well. It’s also about maximising your luck factor when it comes to being assigned tasks, and ensuring you’re not stuck with things you’re bad at.

In a way for juniors, their progress will depend heavily on the alignment of their luck and skillset. Seniors, had more (but not full) control, over how to steer their progress.

This meant that in this case: Luck isn’t binary. You can increase the number of situations that align with you.

How to Maximise Your Situation Luck at 100x and 10x

For everyone, but especially for juniors and those just starting out, the best thing to do is to keep trying new things in technology and exploring. This will help you figure out what you’re good at and what you’re bad at.

For those who have a better idea of what they are good at, the next step is to expand the number of situations where they can perform their best. This involves some practice and research into adjacent technologies to the ones they already know. Once you become great at something, make sure to own it so your managers and bosses know that this is what they should give you. Which will increase the chances of hitting those 10x or 100x situations more frequently.

Find at least one thing you’re good at, even if it’s a very small and narrow thing. Build up from there. (Some notable examples include regex and configuring webpack.)

For seniors, if you haven’t done so already, start looking beyond the technical aspects of development. This doesn’t mean you have to move into management. But with your unique technical knowledge, you can play a more active role in understanding the user or business use case. This will allow you to optimize with your managers to ensure 10x or 100x success rates for you and your team.

For those who are job hunting, if you know you’re good at something, even if it’s 10x, do some research and dig deeper into your job hunt. Try to find a well-funded startup or MNC that is facing difficulty in the exact thing you’re good at. While this can be difficult to pull off and does involve some networking, if you place yourself in the right place and time, it will maximize the impact for both you and the company involved.

These are all things, you can do slowly over time, to improve the luck factor. And deliver that 10x experience more consistently.

Special shout out to read article on how to create luck: https://www.swyx.io/create-luck/

As a CTO and Team Leader Today, This Still Holds - 10x Is a Myth - Everyone Is Both 100x and 0.1x

Some may think that the CTO of a UI testing tool such as Uilicious.com would be good with front-end technologies. However, this is far from the truth. Our average fresh hire or junior in the company is typically faster than me the CTO when it comes to writing front-end components with modern front-end frameworks such as Vue.js or React.

This has changed my perspective as a team leader/manager. It showed me how toxic and bad the 10x engineer myth is. It’s an oversimplification of labels and generalization of very unique situations. It’s a toxic misconception that needs to be fixed.

Almost every startup 10x engineer myth, when looked into deeper, turns out to have an individual with unique knowledge in the right place and time. And was able to avoid the tasks they were 0.1x at.

In my case, it was focusing on infrastructure over front-end development. Where instead, I have other team members who are 1000x times better at doing frontend development.

This is an important realization as a manager because, for every individual who is 100x at one task, there’s a task that makes them 0.1x as they get stuck in a time sink. It’s about understanding and recognizing what each individual is 100x at, and what they’re 0.1x at.

While this sounds very straightforward, this is significantly harder in practice.

There are many things a team leader or developer will not know till they have tried it before. Or are given the time and opportunity, to hone and master the skillsets involved in those specific situations. There are also many things, especially new technologies, that no one is good at it, and is all about taking risks and a chances.

It’s the team leader’s job to organize tasks accordingly to best suit everyone’s needs. And if that is not possible, it's also about understanding that even if someone can’t be 100x in your situation now, they can potentially be elsewhere in other teams.

And is on every individual developer, to figure out what they are good and bad at, and to communicate that to their team accordingly.

So to death with the 10x Engineer Myth, Long live the 100x Engineer in specific situations

PS: If you have not made any new year's resolutions. Perhaps increasing your luck surface area could be it!

~ 🖖 live long and prosper
Eugene Cheah: CTO of uilicious.com

Image credits

* Austin Neill - rockstar image: https://unsplash.com/photos/sxxnM-tRlVg
* XKCD - comic on standards: https://xkcd.com/927/
* CenturiiC - Comic on experienced engineers (in a tweet)
* Lucas Santos - Dice picture: https://unsplash.com/photos/XIIsv6AshJY
* Vlad Hilitanu - Lego figs: https://unsplash.com/photos/1FI2QAYPa-Y

Gonna skip out on writing a full length article this week, and so should you (whatever you are working on).

This is a reminder that despite all that is a rush happening through out your small tech circle. The world at large is much slower.

It took years for the internet, email, or smartphones to be adopted - the same goes for new fancy tech.

So it’s ok to have a one day break or two during Christmas and new year, spend some time doing silly things with friends or family.

Rest, so that you can better pace yourself for the year ahead.

~ Until next time 🖖 live long and prosper
Eugene Cheah : CTO of uilicious.com

Let’s ignore the public hype that is chatGPT for a moment, its ethics or its implications to society, and let’s focus on actual field notes in building an AI based on its research and the tech, for your startup or your portfolio.

TLDR 1: Dataset is no longer a hard requirement

TLDR 2: It has very broad use cases

TLDR 3: Cost and Limitations

If you were on holiday, and have no idea what are ChatGPT or LLM (large language models) - you can find the briefing about the technology here :

So what do I mean in longer form?
Grab a cup of coffee and settle in for a bit…

Part 1: Data is no longer a Hard Requirement

Crash Course On Traditional Text-Based AI Data Requirements

Note: This article is discussing only text-based models and does not apply to image models in many cases (e.g., computer vision).

In the past, building an AI model typically involved gathering a large enough dataset for a specific use case. Different AI model training techniques and tools existed to optimize the training process or its computational costs. However, the size and quality of the dataset was still one of the predominant factors in training an AI.

This would result in a graph something like the following:

All numbers are large approximations and will change drastically depending on the use case, AI model, and data quality. The scales on the axes are intentionally vague and inaccurate because they are subjective. For technical numbers, read the papers in the citations for specific use cases.

The general idea remains, however, where small-scale datasets produce effectively random data before slowly becoming better and finally being somewhat useful at approximately the ~500GB part.

This led to the dataset arms race, occurring in nearly every field of specialisation related to AI (especially anything computer vision-related). Which played out across multiple startups from 2009 onwards until today, with a few notable startups being acquired in part for their valuable datasets (and usually talent).

Because the arms race was constantly escalating, it became increasingly difficult for new startups to compete with their AI models (small datasets) against incumbents (larger datasets).

For example, at uilicious.com (low-code UI test automation), we used our limited dataset to train an AI model (called TAMI v0.1). We found that it was spewing garbage half the time, leading us to set the model aside and grow the company without AI while building up our dataset.

In a very overgeneralised and non-technical fashion, AI trained in this way will be referred to as "Specialised Models" from here onwards.

The Start of LLMs or Large Language Models

In the quest for a truly universal or general-purpose AI, especially in the field of human speech interactions (as humans make for the most random of all variables), efforts arose to train a new type of text-based AI for extremely large datasets of public data (think Wikipedia, Quora, StackOverflow, and half of the internet text).

Because these new models pushed the boundaries of dataset sizes and model size (think brain size), the methods used to build them diverged from specialised models (which tend to have more emphasis on accuracy and efficiency).

Text AI models trained in this way are now collectively called "Large Language Models" (LLMs).

The downside of this approach was huge, preventing its early application in many use cases:

• Extremely large datasets need to be built and maintained (GPT-3 used 45 TB)

• Training costs millions of dollars (GPT-3 cost over $4M to train, GPT-3.5/chatGPT is presumably higher)

• Limited use for specialised use cases due to the lack of private training data

• Lower accuracy than specialised models in specialised use cases

• Expensive to run (more on this in limitations later)

LLM was dominated by Google (who already had the data and had the motive to do so for their AI assistant) and OpenAI initially. Later, Microsoft, Amazon, Apple, Salesforce, Meta, and a few others joined the race.

However, due to the sheer size and cost to train such a model, it's generally reserved for major tech companies with deep pockets.

Though the first few generations of LLMs may have produced disappointing results, as they lost out to nearly every specialised model in every task, things changed as years passed and they scaled up in both dataset size and model size.

Their benefits became more visible:

• Facts and accuracy aside, they became really good at talking to humans

• In many (but not all) cases: they are really good at learning new specialised knowledge when given the datasets in an appropriate format
  (you can't just dump it in)

This made drastic changes to the curves:

It also sparked a new wave of AI-backed business models in 2019. Startups could now train new AI models on top of existing LLMs with datasets within their reach, from chatbots to presentation slide generators, code copilots, copywriting and even D&D gamemasters.

No longer was these AI models in the exclusive domain of large tech companies. For a small fee for training and running the AI on their infrastructure, OpenAI and Google started opening up their models for others to build upon as a means to profit from the LLMs.

This was beneficial to startups, too, as they no longer needed millions of dollars in R&D investment required to build these large models internally, allowing them to reach the market faster with their proof-of-concepts.

Data was still, in a way, king. It may have moved the curve to be more accessible, but it still required a team to build up large datasets and tune the model. As a result, while many of the 2019 startups found it significantly easier to build their prototypes, many found it difficult to cross the "useful" threshold, as they needed to scale up their dataset with diminishing returns.

This is consistent with early internal trials of uilicious.com TAMI AI models - basically it was between interesting and "meh".

But that's when things start to really change in 2022....

How Large Language Models are Few-Shot Learners

The release of GPT3.5 (or chatGPT as it's known on the internet) last month (December 2022) had huge implications for the way AI models are trained and fine-tuned.

It has unlocked the possibility of creating usable AI with extremely small datasets - something that most startups have access to or can create by hand. This is a fundamental shift in how we think about AI training.

At uilicious.com, we were stunned when we discovered that a small sample dataset smaller than 1GB, of our larger ~100GB dataset. When converted and optimised with the new training techniques, had exceeded the "useful threshold" - the point at which the AI could be utilised by our users and outperform everything we had previously.

While subsequent experiments with larger datasets showed diminishing returns. The key takeaway, was how little data was needed to "make a useful product". It took us less time to build a proof-of-concept AI, fine-tuned for our proprietary use case, than it took to write this article.

Using GPT3.5 as a basic building block, it is now possible to build useful AI applications for various use cases without needing a specialised team or individual.

Depending on the use case, the dataset can be as small as a single sentence or paragraph, or up to 100MB or 1GB - a size that is achievable for many startups.

Alternatively, if you can get chatGPT to act in a way that you find useful and of value to your startup. You can build it as a dedicated AI service.

The drastic reduction in required dataset made it possible for us to build a "useful" AI model with only a small fraction of our full dataset - something that had previously been "useless" or "impossible" at our scale.

In many ways data is no longer king (thought still important), finding and building useful application is the true kingmakers with this new AI.

Screenshot is a demo of our AI in action, generating full UI testing scripts, from a prompt, as an example integration.

Something which is now doable in a week, with the right team, and the right data.

Do reach out to us if you are interested in the future of AI and UI testing.

Part 2: The Hunt for Use Cases (and AI Interns) has Begun

The intern framework for finding use cases

GPT-3.5 and all other large language models have limits on their accuracy and reliability. This is partly due to their nature of dreaming out answers.

While in theory (yet to be proven), given a large enough specialised dataset, its accuracy can be drastically improved for specialised use cases. However, if you have such a large dataset, chances are you could already build "specialised models".

Ultimately still, this limit on accuracy and reliability is only a critical issue in sensitive industries (e.g. medicine). For most cases, it is merely a distraction in finding use cases. Once the AI crosses the “good enough” threshold.

A healthier and more realistic approach to the problem is to imagine GPT-3.5 as a remote and paid “smart college intern on demand”

Because realistically, the AI has all the same limitations in such use cases, in addition to being remote and online:

• Never trust interns to be 100% right all the time (let alone diagnose medicine)

• Create a workflow that allows the intern to safely provide useful value, with the appropriate supervision, guidance, and planning

• For one-day or one-month interns, you would create a formal training plan to quickly get them productive in one hour or one week of training, respectively- You should be paying your interns; they are not free (both for humans and AI)

The only real benefit the AI model has over real-life interns is:

• They are not affected by school seasons

• You can scale both up and down at any second

• The training is something you can do once per batch and scale up to as many instances

The downside the AI has over the humans is that they cannot get you coffee in person.

Once you frame it in these terms, it becomes significantly easier to figure out how to integrate the AI into existing business processes or products.

Need to generate article ideas? Get a one-day intern.

Need to convert some proprietary Java to JavaScript code? Get a one-month intern with some basic training on how to do the process.

And so on and so forth - where at no point in the process should the human be removed from the loop in supervising and iterating with the intern.

On a slightly more technical level:

- A one-day intern can be quickly built using prompt engineering; anything you manage to get chatGPT to do fits into this category. The downside is there is a practical limit (about 2000 words) of what you can fit into the training data. The plus side is you can experiment and set this up in seconds, and it's very easy to test with chatGPT.

- A one-month intern (or anything in between) is where you start having a formal training dataset, which they can memorise and learn. They will be able to handle more situations according to the training materials. The downside is you actually need to prepare that training material; and its a more technically involving process.

Part 3: Cost and Business Limitations

Cost per Prompt Will Make or Break the Business Model

This is the biggest weakness for this newer approach in AI, which is based on bigger and better models. Unfortunately, there is no free lunch. While it is cheaper and easier to train for specific tasks in terms of dataset size, it is significantly more expensive to run compared to more traditional AI models.

It is not cheap; the cost per prompt and answer range anywhere between one cent and fifty cents, depending on how much data was needed to train or use in the process. This is drastically higher than a typical API server, which can handle a million requests per dollar.

To put it in simpler terms, it costs more in server hardware to handle one AI query for one user within a given second than it does to serve a million user requests for a typical medium-sized Shopify website.

This is not due to OpenAI or Azure soley trying to make a profit; it is down to the pure server hardware infrastructure required to run such large models.

As a result, as powerful as it is, incorporating such Large Language Model AI has a hefty price tag and may not be applicable for all use cases due to this limitation alone.

The end result is that, while many use cases can benefit from the use of such an AI, not all use cases can afford it; and this should be an important consideration for any implementation.

Example A: Support Service Chat

For a support service desk, a typical support staff can handle ten customers per hour, with each customer having an average of fifteen back-and-forth prompts. If it is five cents per prompt - this adds up to $7.50 s an hour if an AI was used to mimic a single support staff.

This is not only cheaper than the typical US call center staff median salary of $15 per hour, it is also much more flexible (no staffing overhead, can scale up and down instantly).

Likewise, it is also possible to take the same "intern" approach, where these support AI serve only as L1 support, allowing the humans to handle the more complicated cases. In this scenario, it makes sense when done and scaled appropriately on a per-prompt (or per-hour) basis.

Example B: A SaaS Service to Draft Emails

The average office worker replies to approximately forty emails per work day, or approximately 880 emails per month. Even at five cents per email, that would be $44 per month on average per user just to handle the email replies.

What makes it worse is that it's reasonable to expect that, with such a service, an office worker would be able to reply to more emails on average. It would not be unreasonable for the average to double to two thousand emails, or one hundred dollars a month, in pure AI costs alone.

In this case, if the SaaS startup were to do a simple, let's say ten-dollar-per-month pricing, they can get into heavy potential losses over time.

This pricing cost and business model runs counter to the typical, per-user pricing model that is common in SaaS. Which is why it is common for such integrations to have a "credits" system as a means to cap the usage, and a means of billing, of such an AI.

It is expected that overtime, with better fine-tuning, competition, and cost optimisation, the price per prompt can go down. Another notable method is to use the original more expensive AI first at launch, while collecting more data, which is then used to train a more specialised and cheaper model. However, all of these methods go into deep technical details, which may be unique for each use case, and generally require significant time and effort.

And even then, while it can be a drastic ten times in savings, it is fundamentally more expensive than traditional SaaS API services.

OpenAI Has an Effective Monopoly (For Now)

Prices May Be Stuck Until Competition Arrives

While there are existing open-source Large Language Models, in very frank terms, they are either comparable to GPT2 or somewhere in between that and GPT3.5.
For some simple cases, once they start building up a reasonable dataset, these smaller (and cheaper) models may be useful to migrate into as a means of cost cutting.

However, for other complex cases, such a move may not be possible due to the complexity of their AI, giving OpenAI an effective monopoly with no incentive for lower pricing.

It is believed, however, that over the course of the next one or two years, the open-source community will catch up and, in doing so, perhaps allow prices to improve due to better alternative infrastructure providers. However, because this is an uncertain future, it is worth highlighting.

OpenAI Is a Potential Unwilling Competitor for Some Models

While not intentionally, it is critical that startups within this space build feature sets that are defensible beyond their prompt-to-text bots.

For example, there were several smaller startups who built GPT3-based bots or SaaS around specific prompt-to-text use cases, like name generators or idea generators with a simple interface.

Literally overnight, with the launch of chatGPT, these small single-use-case text-to-text SaaS were made redundant, as everyday folks can now get the same functionality via chatGPT for free. While it may not have been OpenAI's intention to compete with the very partners that build on them, it may be its inevitable nature to do so, as they keep improving their model and chatGPT.

As such, to ensure this does not repeat itself, it is critical for any business model around this technology to figure out what additional value they provide beyond just prompt-to-text, perhaps better user experience or integrations with existing toolings, etc.

Lastly: It Ain't Perfect

Reminder of the intern model, do not expect to use this for curing cancer tomorrow. So please do not go stuffing AI into every product and startup on Earth when it makes no benefit to the end user to do so.

~ Until next time 🖖 live long and prosper
Eugene Cheah : CTO of uilicious.com

Additional notes that didn't fit in

swyx also does an excellent job in trying to consolidate various information, in this rapidly chaotic and growing space, which is very worth reading

• The Day The AGI Was Born: https://lspace.swyx.io/p/everything-we-know-about-chatgpt

• How Open Source is eating AI: https://lspace.swyx.io/p/open-source-ai

• His live AI notes: https://github.com/sw-yx/ai-notes/blob/main/TEXT.md#chatgpt

Currently BLOOM is the main opensource contender for GPT3 (not 3.5): https://towardsdatascience.com/run-bloom-the-largest-open-access-ai-model-on-your-desktop-computer-f48e1e2a9a32

In the AI / ML industry, the ability for a LLM to quickly learn new concepts and apply them is quantified and measured by tests called "Zero-Shot", "One-Shot" and "Few-Shot" learnings.

Generally the better the AI does in these test, the less data you need to train it for your use case.

In my opinion: This make sense in hindsight - who would have thought then a Neural Network modeled after humans, would act like humans. And benefit from the T-Shape education model. Where a large breadth of generalised knowledge, help improve ones ability to learn specialised knowledge in one are of expertise. (this statement is not backed by any data)

Other sources and citations

• GPT-3 45TB training set: https://www.springboard.com/blog/data-science/machine-learning-gpt-3-open-ai/

• GPT-3 estimated $4 million++ plus training cost: https://lambdalabs.com/blog/demystifying-gpt-3

• Bloom AI, an opensource competitor $1.6 million training cost: https://techcrunch.com/2022/07/12/a-year-in-the-making-bigsciences-ai-language-model-is-finally-available/

• T-Shaped model of education and work

  • https://wordspy.com/index.php?word=t-shaped

  • https://collegeinfogeek.com/become-t-shaped-person/

• Large language models papers, for zero and few shot learnings

  • https://arxiv.org/abs/2205.11916 (zero shot)

  • https://arxiv.org/abs/2005.14165 (few shot)

• Call center salary rates in US : https://www.comparably.com/salaries/salaries-for-call-center-agent

• Average email sent by an office worker a day: https://www.templafy.com/blog/how-many-emails-are-sent-every-day-top-email-statistics-your-business-needs-to-know/

• OpenAI announcement of chatGPT / GPT3.5 : https://openai.com/blog/chatgpt/

• Unsplash photo credits

  • https://unsplash.com/photos/7swaW1bYpWI

  • https://unsplash.com/photos/1iVKwElWrPA

  • https://unsplash.com/photos/SELXIJwN24s

  • https://unsplash.com/photos/3xwrg7Vv6Ts

  • https://unsplash.com/photos/-G3rw6Y02D0

This article focuses more of the current state of AI, on 5th Dec 2022. As a means of briefing others outside this industry, what has been going on recently, and what are its limitations. With a focus on recent developments in chatGPT, over long term speculations.

For those in the tech / AI industry, if you need an article to explain what is going on to your non tech friends, this article is written for that use case.

Recommended music with the article - GITS Stand Alone Complex: Be Human
@youtube-music @spotify

In 2022 artificial intelligence (AI) has made some major strides in its capabilities. Largely in the two following area

• Prompt to text generation

• Prompt to image generation

Collectively, these "prompt-to-output" models are designed to take a prompt (or a question) and generate a finished product in either text (like a clickbait headline, or programming code) or image form (an illustrated art image, or stock images) based on that prompt.

This type of AI is incredibly powerful due to how simple it is to use, and has the potential to revolutionise many different fields, and can literally feel like magic.

An example would be how a single prompt like

Hey, I'm trying to come up with some interesting, fantastical way of decorating a living room for a design project

Can be used to generate the following image

Or write business plans for a website

Or even be used to help generate entire novels

Because its full scope of its capabilities really requires you to try it, to better understand the potential for yourself. I would highly recommend giving these services a try, while its still free at https://chat.openai.com/chat
(it may turn to a paid service in future)

Holy Crap, can it generate anything?

Yes, but that includes dreaming up lies.

In an extremely convincing fashion that even experts can find it hard to tell the difference.

It can also be very stubborn in how it present itself, so as hard as i tried to get it to help me write this article, unfortunately it just couldn't be "me".

In a weird way, you should view these AI as unique assistant who ...

• is a naive smart kid

• who is slightly high or dreaming in the clouds

• have very unique and flawed views

• and is very willing to bluff their way, in providing a response, even if they dun have the answers

If you ever dealt with such an individual, you know that there are random nuggets of pure gold wisdom that they will say from time to time, and bulls#!t you will need to filter, as it dreams up the answer.

The later act of filtering, is potentially the biggest danger of this new technology.

As it can be very very difficult, because of how convincing the AI structure its answer, if you are not an expert in the respective field.

For example, as an experiment, I asked it questions in domains i had zero (or even negative) expertise, like cooking or chemistry. And I realised I have nearly no capability, in validating its bluffs.

This is despite my best attempts to google and fact check it at times, where it may give sometimes answers that is slightly different, but I have honestly no idea if its right or wrong. Unless I try it myself.

Which can potentially be dangerous, seeing as how it can get things wrong in my area of expertise.

How is that possible, wasn't it trained on nearly a terabyte of data?

Yes, but it is not thinking, it is dreaming.

To elaborate, let’s start with an oversimplified explanation on how these AI models fundamentally work.

First the AI is given a large dataset of articles or images from various sources, it uses this data to provide the foundation of its knowledge, loosely connected together. This is then trained together with possible prompts and responses given by the user.

Once this is completed, we have a working neural network or trained model with all this loosely connected data. Frozen in time.

When a prompt or question is given, the AI is not thinking of an answer or deliberating from the trained model. In a way, it is hallucinating the answer using the snapshot of knowledge it has.

Now, lets use a human analogy instead

The human analogy, would be your brain right now, with all your past knowledge, or experiences in school or in life. Snapshot and recorded, in your head. Minus your emotions, and your feelings, or anything hormonal related.

When a prompt is given, imagine it as being told to you, before going to sleep into a dream. For example "go buy milk tomorrow, when you wake up".

For most folks, sleep is when they as an individual have little to no control with how the story goes (ignoring the lucky few lucid dreamers).

In this state, your mind in its dream state, starts to take that last phrase that is stuck in your head, and starts dreaming a story around that phrase.

Where you are an individual inside a dream, and your brain the dream world doing its best to be as convincing as possible that its real and not a dream world, continuously writing the story back and forth with your interactions and actions.

All while using the snapshot of all your past knowledge and experience, in your brain. To make your dreamworld as realistic as possible

Flipping it back around

You, with your prompts is a participant in a dream.

With the AI being the dream world, trying its best to be, as convincingly as possible, a good dream. Where it answer your questions in a very believable manner.

And just like your dream world, it can be at times entirely realistic, and factually accurate, or as imaginary (and perhaps exciting) as it wants to be

As such, this dreamlike state, allows it both to make basic math mistakes with high confidence

To hilariously answer questions as a cat when instructed

To very impressively, even emulating interactions with a virtual machine (rather accurately!)

What does it mean for the future of work?

It's difficult to predict exactly how the advancements in AI will impact the future of work, but there are some potential scenarios to consider.

One thing that seems clear is that the current wave of "prompt-to-output" models is having its "iPhone revolution moment". Just as the iPhone changed how everyday people interact with software and ushered in the era of smartphones, prompt-to-output models have the potential to change how we work and interact with tools.

It has already started fundemnatlly changing industries, today. Especially in SEO copywriting, and stock illustration / photo industries.

The next wave could manifest in waves of AI assistant work tools and practical enhancements to existing tools, ultimately making automation more accessible and lowering the barrier of entry for performing tasks.

Regardless of the forms of changes, this is a significant development and every startup founder (including me) and product owner should be taking note of how to integrate AI into their business. Just as the smartphone/iPhone era made entire industries and tools obsolete and created new domains of app-based tools, the rise of AI has the potential to do the same.

However, it's important to note that not all tools will be impacted by AI. Some industries, may not see much change beyond a new skin or UI. Only time will tell which side your business falls on.

PS: if you are feeling FOMO, and is in the tech industry, chill - the iPhone and app revolution happened in months and years, not days. Do go on with your life, and your holiday plans.

As long as you are not ignoring the potential changes, and making steps towards it - you are good.

If you are outside the tech industry, keep a look out, be ready to learn new tools, across the next few years - that may change your industry. And be prepared to learn and use them to your advantage.

~ Until next time 🖖 live long and prosper

If you ever wanted to know how it feels like to be a senior software engineer, and experience ...

• The joy of coming up with experiments, and testing them to solve problems

• The craftsmanship of building either building small hyper-efficent systems, or

• The pride in building large hyper scale systems.

• All within either a fixed constraint or the lack of it

Factory building games, allows you to experience all of them in a highly compressed timeframe of hours instead of days.

All without the common frustration and pains that comes with programming. (Weird obscure library bugs, lack of documentation, Company politics or bureaucracy in processes, budget ... )

What is a Factory Building Games?

Factory Building Games, or FBG for short, are at its core about
Constructing and expanding a large base of multiple building blocks …

• Connected together to build products using an ever-growing continuous supply of materials …

• To continuously feed a very intensive "resource sink" (the games goal) …

• Which runs automatically without the player interaction for long periods of time

One of the key differences between FBG, and typical resource management games. Is the endless continuous nature of the main production lines, due to the usually infinite scaling nature of the "resource sink". Allowing everything to run "infinitely" without player interaction.

Said "resource sink" may differ from game to game, but it's typically some combination of the following

• Researching new tech, some which are endlessly repeatable

• Sustaining a colony of citizens, that grows in both size, and resource requirements

• A definitive end game goal, with enormous requirements (launch a rocket, build a Dyson sphere, etc)

While this may sound simple enough. Another common feature, is how as the game progresses. It would require more advance products and buildings, that requires other products built by the factory system itself. Which constantly scales up complex combination requirements.

At late game, it is not uncommon for products that are needed for the game goals, to require a process that involves over a dozen parts, and hundreds of steps.

Resulting into a complicated large scale processes, running continuously; Which mirrors some real life factories, hence the FBG title.

This gradual ramp up in complexity encourages constantly scaling up both consumption and production. All while constantly refactoring or updating the factory layout for optimisation, as new "methods of production" is unlocked.

How it makes you a better programmer - by making you practise everything else but code!

To understands the similarities between software development as a programmer, and playing an FBG as a gamer. We start by looking at a typical workflow for a programmer, where each task spans weeks to months

Now contrast to the typical gameplay experience of a FBG.

Notice the huge similarities, and how it compresses all the programmers day to day activities down to hours. While nearly removing all the unpleasant and potentially frustrating bits!

Despite ironically removing coding (one of the most critical things a programmer does). It rapidly speeds up the loop of planning, testing and iterating. At a much faster pace than the real world.

This allow a player to practise systems thinking, which is critical to programmers.

It does so, by forcing the player, to split the problem in game into smaller tasks, while using their existing tools and knowledge to solve them. Before forcing them to do the inverse, in putting together large complex systems, using the same "blueprint" of what they built previously.

This process of making smaller changes, and integrating such changes to the larger system, is part of the day to day of software engineering. (including all its failures and iterations)

Additionally, due to its nature of being a "game" - and the lack of "real life consequences". Most of the negative emotions that come with failure are minimised. This encourages the player to learn quickly through experimentation and failure.

All while avoiding overwhelming the gamer, due to the limited nature of the toolbox in the game. Allowing the player to gain satisfaction, from the mastery of said tools

Hence, despite the exclusion of code. It teaches the gamer/programmer, very essential skills in software development. As it’s worth repeating that …

There is a flip side though - which should come with a warning label. This game genre can be very addictive, due to how well refined the positive feedback loop is.

"It's the one video game (Factorio) that everyone at Shopify can expense”
~ Tobi Lutke, CEO of Shopify

Sounds good, how can I get started?

You can find the full list of FBG games at :
https://github.com/PicoCreator/awesome-list-factory-building-games

However, because the full list can be overwhelming, here are some quick highlights …

Dyson Sphere Program : which is a pure FBG in space, with really nice 3D visuals. Also building a Dyson sphere around the sun, is a cool mega project in itself

Satisfactory : which uses FPS/Fortnite like controls, not recommended if you are not used to such a control scheme - supports multiplayer.

Factorio : which is the most popular in this genre - is only recommended for more experience gamers. In my opinion it's not beginner friendly. However it supports really large scale multiplayer gameplay (50+ players), making it popular with larger teams.

Autonauts : is one of the notable FBG that includes simple programming blocks, for its "automation bots". Adding the code element back into FBG. It does loose however the iconic conveyer belt (to force you to use the bots), while sadly simplifying many things (can be too easy)

If you find yourself struggling with a particular game, try to takes things slow instead of rushing it. Part of the fun is to overcome these challenges.

Also, Some FBG may include other game mechanics like endless combat/war. this can drastically change the style of the game, as such when possible I would recommend "peaceful mode" for new players. To focus on the FBG gameplay.

I did not like the games you recommended, does that make me a bad programmer?

No

There are many good programmers who dun like games

There are many other reasons why one may find joy in programming that is not found in these games.

Most notably

• Personal satisfaction from providing positive impact to users

• Personal satisfaction from solving coding tasks

Similarly there are many other valid reasons why you may dislike a certain part of the game which makes it a dealbreaker. Or even just video games in general.

So while there is some truth that if you enjoy FBG, you probably will enjoy programming (and vice-visa). It is worth reminding that correlation is not causation.

So you be you. There are other hobbies in life, which is both fun, and sharpens your skills.

However, for programmers, who do enjoy playing such games. Good for you, as this is one of the few games which can help both practise your skills, and have fun outside your work-hours.

Separately, For non-programmers who is aspiring to be a programmer, completing an FBG, will help you get a better idea of the ideal workflow of a programmer.

This in itself, is a huge benefit, if you are outside the tech industry, and want to explore if such work is suitable for you

Shameless plug: How can Uilicious.com automate cross browser testing, to make my website development more "Satisfactory"

Is manually testing your websites, prior to production deployment a huge time sink and pain? Find existing website testing tools too complicated for your team to use?

Give uilicious.com a try today, with our easy to use testing commands, which anyone, even manual testers can easily learn.

An example would be the following code snippet for a simple login test

Which would provide the following test results

Making testing user journeys on the internet, a piece of cake 😄
~ Eugene Cheah CTO / founder of Uilicious.com

Beyond specific tech tools or programming skills, these would be

• Systems Thinking

  • Understanding processes in logical terms : If this happen, do this, else that

  • Understanding of "cause and effect"

  • Planning and deciding on how to change processes, to reach a desired goal

  • Practising cause and effect on a much larger scale. : Visualising, organising and understanding how large systems and processes, interact with each other over many logical interactions. Especially how any unexpected outcome in one system, can cascade into various errors in other systems.

• Other very useful skills (to help with systems thinking)

  • Keeping track of a large number of things in your head

  • and/or organising a large number of things on a board / paper / chart

  • Simple math (plus, minus, multiply, divide).

  • Advance math and simple statistics is a huge plus (but not a requirement).

  • Some imagination and creativity

• Soft skills (that universal to all jobs)

  • Communicating and coordinating within a larger team

  • Dealing with stressful situations

If the items in “Systems Thinking” looks familiar, it’s because you are already practising it in your current job, or daily lives, even if you do not realise it.

Almost every job involves either executing a process, or planning one, as part of a larger system.

For example, a chef in a restaurant, will check the inventory of ingredients that they have, at the start of their work shift. With the following in mind …

• if any of the supplies are low, they will need to make an order for ingredients

• if any of the supplies are low, and the ingredients will not come in time. They may need to remove items from the menu

• failure to do so, may cause orders to not be fulfilled later in the shift, which may then cause unhappy customer

• unhappy customers, may cause unhappy bosses, or loss in revenue/tips.

So the process in this case (inventory checking), is straight forward and is done once per day.

Its purpose is easy to understand (maintain business operations)

Its failure is understandable (unable to maintain cooking process). - Its failure even has cascading effects to other systems (waiters needing to manage the chaos when things run out with unhappy customers).

You may even be thinking on how to improve the processes: For example you are able to cut down on serving time. If the waiter’s uses a messaging system to input their orders in a tablet, an order is automatically printed out in the kitchen at the other end of the hall.

Thinking about these processes, and how it can be improved or changed; experimenting with it, and understanding how such changes can improve (or degrade) the process.

Is all part of systems thinking. In a way …

❗ Programming is defining and iterating the day to day processes (called functions), on a whole different mind-bending scale of executing a billion times over.

Instead of planning and executing a process which is done once a day, it’s executed potentially a million times a second. Likewise, instead of a handful of different documented processes in a single restaurant, you are looking into potentially over a hundred thousand different functions and processes interacting with each other.

If the scale of these functions and process sounds intimidating, worry not, because you will only be building these things things, one function at a time. While working with other developers, in building and interact with their functions.

You do not need to understand the working of every process and function, you would only need to have a high level general abstract understanding of it all. And just enough understanding to use other peoples functions.

All while maintaining a deep understanding of the function and process you are writing.

Large teams is also where soft skills in communications, is essential in more senior positions, with larger team sizes.

If thinking about processes, the why’s and hows, everyday and improving on them appeals to you, this maybe the right job for you.

Especially, if you find joy in helping others lives or quality of life improve through the processes you put in place. Programming is a path that is able to let you do so at scale.

If all this sounds good, perhaps programming is a career that is suitable for you

As a programmer myself, there is a danger that the above list of skills can be viewed as a form of gatekeeping. But that is not the intention here. If you are weak in any of the above core skills, it does not mean it’s impossible to be a programmer, or to learn programming.

It simply means you have to start practising and developing these skills today. And figure out a workflow that helps compensate for the gaps. For example, if you find it hard to visualize in your head a large number of things, put it in writing, put it on pen and paper.

This article is part of larger series of "Questions from devs to CTO" - and my attempts to formalize my verbel answers into a series of articles.