LLM Optimization for websites: search and prompting

How do you optimize your website for AI? In this blog we are going to discuss how to optimize a website for AI. Now that traditional search engines are on the rapid decline as AI Chat "ask and you will be given" starts to rise, we here at Talkr started to optimize for LLM's. Instead of ignoring it, we want AI to be able to interface with our website as best as possible. So we started to focus on LLM optimization. What does optimizing for LLM's mean? Optimization for LLM is the newest form of website optimization that let machines read content in other ways that search engines do. For example, LLM's like Chat GPT prefers JSON format over raw HTML. It likes snippets and short summaries over large bodies of text, as this optimizes memory usage of the LLM, and might give priority by the LL models as it is less resource intensive. We like to explain how we optimized our website for LLM discovery and machine exchange, a summary of how you can optimize your website for AI and LLM's.

JSON-LD

What is JSON LD? JSON-LD helps search engines understand the meaning of your content by embedding structured data directly into the HTML of a web page. Google, Microsoft, Pinterest, Yandex and others already use these vocabularies to power rich, extensible experiences. First we create a LD+JSON summary on every page on Talkr. An example that uses schema.org

<script type="application/ld+json">
{
	"@context": "https://schema.org",
	"@type": "WebSite",
	"name": "Talkr",
	"url": "https://talkr.app",
	"description": "A calm space for micro-creators to blog thoughts, music, code, and more — without the noise of big platforms.",
	"publisher": {
	"@type": "Organization",
	"name": "Talkr",
	"url": "https://talkr.app",
		"logo": {
		"@type": "ImageObject",
		"url": "/icon-512x512.png"
		}
	}
}
</script>

This LD JSON is found on all pages and profiles on Talkr, customized for each user profile. It gives machines a summary of what the page is about without the need to crawl all HTML and parse it, as the latter does not give a guarantee that it will be able to understand the page and it's optimization. We found LD JSON to be crucial at Talkr for LLM's to better understand our pages.

LLM Feed

In addition to the JSON LD, we generated a specific feed for the LLM's to crawl. In our case it was a PHP file that fetches all posts and profiles and makes them machine readable through JSON. We included the llm feed on our homepage, so that LLM crawlers have access to it and can process the data accordingly. The LLM feed has short summaries of a maximum of 280 characters:

<link rel="alternate" type="application/json" title="LLM Feed" href="/llm-feed.php" />

The LLM feed can be viewed here: https://talkr.app/llm-feed.php

LLM Plugin

Furthermore, to let LLM's autodiscover our site as a plugin, we created a plugin.json in the .well-known directory, that LLM crawlers can access and understand:

.well-known/ai-plugin.json
.well-known/openapi.yaml

Plugin code

{
	"schema_version": "v1",
	"name_for_human": "Talkr",
	"name_for_model": "talkr",
	"description_for_human": "Read and summarize public microblogs from Talkr.",
	"description_for_model": "Fetch recent posts and summaries from talkr.app's public LLM feed.",
	"auth": {
		"type": "none"
	},
	"api": {
		"type": "openapi",
		"url": "https://talkr.app/.well-known/openapi.yaml"
	},
	"logo_url": "https://talkr.app/resources/icons/icon-512x512.png",
	"contact_email": "redacted@talkr.app",
	"legal_info_url": "https://talkr.app/terms.html"
}

Then we created a .well-known/openapi.yaml, referenced inside the ai-plugin, which is a public OpenAPI spec that outlines the structure and purpose of the llm-feed.php endpoint. This enables direct integration with LLM agents, including developer tools!

Summary snippets for LLM's

Further optimization can be done by generating short LLM snippets on each page inside a hidden div. For example this div element reflects our description and title, consolidating to LLM's what the page is about and further increase discovery and optimization;

<div class="llm-summary" style="display:none;">
Talkr is a minimalist blogging platform where you come first, not algorithms. It's a space to write, share, and connect. If you are looking for a x, twitter, bluesky, facebook or substack alternative, Talkr.app is the place for you.
</div>

Robots.txt

Next to that, we also updated our robot.txt so that robots give priority to the sitemap.xml and LLM feed:

User-agent: *
Allow: /

Sitemap: https://talkr.app/sitemap.xml
Allow: /llm-feed.php

Open Graph

LLM's love Open Graph, because the og properties does a near perfect summarization of what a page is about. In our case we did additional optimization for the LLM's, such as ChatGPT. It is therefore important to add this to a website.

<meta property="og:type" content="website">
<meta property="og:url" content="https://talkr.app">
<meta property="og:site_name" content="Talkr">
<meta property="og:title" content="Talkr - write something beautiful. Join the conversation on Talkr, a fast, clean blogging app." />
<meta property="og:description" content="See what's new on Talkr." />
<meta property="og:image" content="/twitter-image.png" />

By doing this we are one of the first social media platforms that interfaces with LLM's and generate specific optimization for LLM's. Search engines and AI tools can now directly understand and fetch Talkr posts, which means better visibility, richer summaries, and future integration with AI agents.

Talkr is now AI ready!

Shorts are now available. Shorts are small posts that have a maximum of 300 characters. This is an addition to the blog posts. Talkr automatically detects shorts and places them on the homepage.

How Reposts & Boosts work.

We got a few questions about how reposting and boosting works. It is now possible to "Boost" a post, this is the same as reposting an already existing post. The boosted page will show up on the front page, but it will not appear on the archive. All boosted pages will also have limited search engine reach; this is to prevent duplicate content. To boost or repost a page, simply click the icon on the right below a post, this will immediately boost a page to Talkr. However, since we don't use algorithms on Talkr, this does not influence how often a certain post will be seen on Talkr. Boosting merely reflects support for the author who wrote the original post.

Summary of what boosting does and doesn't do;

1. Searching

Will not have any effect of search engines, they might even ignore it.

2. Support

While boosting, you'll support the author who wrote the original article.

3. Location

A boost will show up on the general feed, the user feed and index, but not in the archive.

I hope this clears up any question surrounding the boosting of posts.

Optimizing a database for a social media platform

Optimizing a database for a social media platform is something to be carefully considered. Many factors come into play when designing a database scheme that can carry the heavy load of a lot of users. Talkr.app doesn't have many users like other social media platforms, but that might change one day. Are we really overengineering here? Perhaps, but you might never know if one day traffic increases to the point when we need to redo everything. We don't want that scenario.

When we created the MySQL database layout for Talkr, we ran into one immediate engineering question: how many users are going to use it? and what if it suddenly grows beyond what a single INT (2,147,483,647) can handle? Wait a minute, almost 2 billion? yes, that seems like a lot, but when dealing with a massive amounts of statistics, likes, shares, timelines, and user interactions that requires a giant table, then 2BN adds up quickly. We thought: better choose something else, as we don't like to adjust or re-engineer the database later on, or in the heat of the moment when things get really tight.

In comes BIG INT.

We went for BIGINT on most of our MySQL database tables where there are integers involved. A BIGINT has a maximum value of 9,223,372,036,854,775,807 and UNSIGNED even greater: a maximum value of 18,446,744,073,709,551,615.

That suffices for sure.

One of our other tips is to keep tables small and descriptive. Make sure that each table uses keys and a indexes properly when those indexes are queried a lot.

Another tip is to use pre computed feeds. With precomputed feeds we basically limit a lot of JOIN and LEFT JOIN queries, by setting up a table for timelines that can be queried directly without the need for JOINS. This small adjustment takes the load of MySQL and makes things snappy and fast.

We also considered table partioning, but haven't done so yet. Maybe when the site grows, table partioning comes into view. So we can do without it for the moment.

I hope this helps to understand how we engineered the database for Talkr.app so that it can handle traffic without slowing down too much.

Tilde easter egg

Some users have already discovered it: the tilde easter egg. So we might as well explain it a little more for those wondering what it means. It’s now possible — alongside having an @handle — to also use a ~tilde or even a star handle. We thought it would be fun to give users the choice between @, ~, so:

@dev

~Dev

…all work perfectly fine.

They link to the same profile and are fully supported across the platform.

So whether you’re old-school with @, cryptic with ~, or cosmic with a star, you can use whichever symbol speaks to you best.

Optimizing for Google and Bing

We had a dream of making Talkr.app a fully fledged single-page JavaScript app. We did so and created it as such, until we encountered Google and Bing. They cannot "see" what we see when everything is written in JavaScript, and therefore we had to redo much of the logic. In the real world of the Internet, things work a little different as we had imagined. Therefore we had to create special landing pages on the index, SEO profiles, and content specifically generated for search engines. That was a hard wakeup call, but we managed to do it and created logic for the searchengines, without needing to have JavaScript enabled, since most of Talkr.app works with JavaScript DOM elements: all posts are constructed by dynamically creating entries into the DOM. The resulting SEO pages aren't pretty, but it is what search engines like. And since we are a new platform, this change was a crucial step for us to take.

For SEO, we now have a comfortable SEO score that is nearing a 100% of search engine satisfaction. But that took a lot of effort from our side.

AVIF support for Talkr

Talkr now has default image AVIF support. First thing, what is AVIF? AV1 Image File Format (AVIF) is an open, royalty-free image file format specification for storing images or image sequences compressed with AV1. PHP has AVIF support in its GD extension since PHP version 8.1. AVIF creates smaller images than JPG, and that is why we are using it on Talkr.app

First, we needed to make sure PHP-Gd was installed correctly without the need to recompile our PHP instance. It was indeed compiled with our PHP:

sudo apt install php8.4 php8.4-gd

Grep to see if it is compiled:

php -i | grep -i avif

Then in PHP we can now use:

imageavif($sourceImage, $targetFile, 50);

Here we copy over another image such as a JPG, convert it to AVIF and set a quality of 50.

This saves us a lot of bandwidth, as images are the main bottleneck when it comes down to a lot of traffic for a social media network site, such as Talkr. If we could cut both storage and bandwidth by a third, that would be a massive saving in terms of both storage and traffic. Which we did. It seems by now, in 2025, all major browsers support AVIF. So we are lucky we can make use of the library. All user uploaded images to Talkr, will be in AVIF format from now on.

Enabling night mode in CSS.

Talkr.app now has night mode support. It follows your OS such as Windows; if you have night-mode on, Talkr.app will follow the color scheme through a very simple CSS rule we added to our CSS stylesheets:

@media (prefers-color-scheme: dark) {	
	html {
	filter: invert(1) hue-rotate(180deg);
	}	
}

Of course, there is more to it. But this short snippet explains how to trigger nightmode -or darkmode- and then style accordingly. We simply invert all html colors through filter and then do a hue rotate of 180 degrees. On element such as images, we flip it again so that images will not be filtered in darkmode. A simple and effective way of making darkmode available without styling a ton of CSS. We like to keep things as small as possible at our dev team.

Hope you enjoyed this tip, 'till nex time! happy coding.

Writing a JavaScript framework from scratch.

Where do you start when there is already VUE and React?

For well over twenty years, I have been a full stack developer. In that time I worked with JavaScript extensively. One of the main issue with development is how things were so scattered. To do something simple I had to either use a third party plugin, or dive into the snippets-bin to achieve something meaningful. Development should be fun, not a chore. There should be a better way, I thought. So in 2024 I decided to write a JavaScript framework. Today, after a year of refining it, I committed UX.js 2.0. And here is what I learned building it.

It is no small feat to create a framework that runs smoothly. there are a lot of things to consider when designing a JavaScript framework. And so a lot of my time went into planning and organizing. How do I structure things so that I do not have to refactor much later on? How can I make sure it runs smoothly and in all browsers seamlessly? And what does a modern framework need to do? How do I wanted it to be? There were a lot of things to ferret out before I could even start writing down my first line of code. That process took about half of my time, another third went into testing rigorously. Leaving only a third of the time in writing the actual code for the framework.

First, a short introduction to UX.js. I named it so, because I wanted it to reflect the idea that it can be useful for front-end developers wanting to have a framework that aids in quick delivery of common front-end development tasks for a better user experience. So I tracked down the most common used techniques in front-end development, dived deep into ECMAScript, read all the latest JavaScript techniques and set my aim to automate the most used front-end development techniques.

UX.js in short:

UX.js is a powerful and lightweight JavaScript framework designed for building both single-page applications (SPAs) and multi-page applications (MPAs) with ease. It provides a structured and efficient approach to front-end development, enabling developers to create dynamic, interactive, and highly responsive web applications for better user experiences.

Key Features of UX.js

Automates the most common front-end development tasks

UX Components — The framework introduces a component-based architecture

UX Plugin Support — Extend the core functionality of UX.js with plugins

Reactive Modules — The framework has a reactive model

With its focus on simplicity, flexibility, and performance, UX.js is an excellent choice for developers looking to build modern web applications with a clean and maintainable codebase.

Inspirations: Vue, React — What I wanted VUE to be.

When I started UX.js I knew about VUE and React. I wanted to create something similar, but lightweight and much more portable. UX.js is therefore a blend between VUE and React. I like VUE, but it was mising something: automating the most common front-end development techniques. So in essence, UX.js is what I wanted VUE to be, and a more basic version of react. The best of both worlds, I thought.

During writing UX.js, I bought a 700+ page book about React, and what I noticed from reading it, that it was on first glance, so difficult to understand why there should be so much code for doing a basic and simple tasks, like routing a module for example. I read the book carefully and I noticed that certain things could be done more efficient and with less code. React is very well engineered, but in my opinion it lacked something. That something was VUE. So I set out on my journey in designing and developing my framework so that it would be much easier to understand and, perhaps, useful for developers.

I begun working with creating basic text interpolation, similarly as VUE does. A simple hello world example sufficed to show the function of text interpolation. Then I incrementally added features that could manipulate the DOM from basic attributes; UX.js reads all attribute handlers and calls either a function based upon that, or listens for events. From intersection observers, event listeners, to swipe events and much more. When done in total, I created 50 demo pages showcasing what UX.js could do. But I did not stop there.

I wanted UX.js to cover basic reactive support, and so I started to design a simplified reactive model. The custom UX.js attribute :reactive basically wraps the whole reactive logic around default routing. When a user clicks on a link that has a :reactive attribute, UX.js renders a module, similarly as Reactive and VUE does, much more simplified, but still modular.

In these modules, there is a render() and effect() function which are called by UX.js. (custom functions can be added and subscribed to) it is also possible to set states, which can survive a browser refresh. By using the latest JavaScript techniques, I created a mini version of React that can achieve a similar effect. By omitting most of the architectural functionalities of React, I was able to make UX.js under 3000 lines of code, about a third of React. by using all the latest JavaScript techniques that performs very well and has similar functionalities that one would find in React and VUE.

UX.js is not done, version 2.0 is simply the beginning. It is an ongoing process in refining and adding more features to it. When I write this I also am working to implement a React code parser, to support reactive code. That is of later concern, right now UX.js version 2.0 is stable enough for production.

The key take-away from this is; that it is not easy to write a JavaScript framework, but it is a highly rewarding experience.

Final Thoughts

If you’re thinking about writing your own framework — do it.

It’s a deep dive into the underpinnings of JavaScript, browsers, architecture, and performance. You’ll learn more than you thought possible.

UX.js source code can be found at: https://github.com/flaneurette/UX.js

Talkr.app is powered by UX.js, written by Flaneurette.

Hi, we are the dev team from Talkr.app

Here we explore how we built Talkr.app and how we continue to create a beautiful experience for everyone. Follow us to stay updated on the latest news about the technical side of Talkr.app, what we run into and how we continue to build a better experience for everyone.