HN Reader

Doesn't it all come down to "what is the ideal interface for humans to deal with digital information"?

We're getting more and more information thrown at us each day, and the AIs are adding to that, not reducing it. The ability to summarise dense and specialist information (I'm thinking error logs, but could be anything really) just means more ways for people to access and view that information who previously wouldn't.

How do we, as individuals, best deal with all this information efficiently? Currently we have a variety of interfaces, websites, dashboards, emails, chat. Are all these necessary anymore? They might be now, but what about the next 10 years. Do I even need to visit a companies website if can get the same information from some single chat interface?

The fact we have AIs building us websites, apps, web UI's just seems so...redundant.

A HUD is an even more "confident" display of data than text though. What do you do with a HUD that hallucinates? Is there a button on each element that shows you sources?

On this topic, can anyone find a document I saw on HN but can no longer locate? A historical computing essay, it was presented in a plaintext (monospaced) text page. It outlined a computer assistant and how it should feel to use. The author believed it should be unobtrusive, something that pops into awareness when needed and then gets back out of the way. I don't believe any of the references in TFA are what it was.

We have explored that sort of debugging/visualization tool in https://pernos.co. We built it before the age of genAI, but I think for coming up with powerful visualizations AI is neither necessary nor (yet) sufficient.

I'd rather flip this around and be in a fully immersive environment watching agents do things in such a way that I am able to interject and guide in realtime. How do we build that, and build it in such a way that the content and delivery of my guidance becomes critical to what they learn? The best teacher gets the best AI students.

> "non-copilot form factors that more directly extend the human mind."

I think Cursor's tab completion and next edit prediction roughly fits the pattern, you don't chat, you don't ask or explain, you just do... And the more in coherent your actions are the more useful the HUB becomes.

All I have ever wanted in my life is a heads-up display for my eyes that puts floating icons over other people telling me their name, birthday, the names of their kids, and what we talked about the last time we met.

There is a giant confusing tech stack here, but opening up this simple plain text page resulted in like 100% CPU usage from from gnome-shell when visiting from firefox on a pretty non-customized Ubuntu 24 install.

In the firefox task manager nothing really looked odd, but opening that tab and displaying it is insanely CPU intensive.

Pausing the autoplaying video makes it seem like a sane web page in terms of CPU usage. I'm surprised how much CPU playing that video consumed.

The Ink & Switch guys always have the best-est of ideas!

Makes me think that I would love to have an AI tool that not only allows autocomplete but also provides suggestions while I write code. "Have you thought about using useCallback here?" "This feature seems already provided by library X and Y" "It seems like this code already exists in function Foo. Maybe a small refactoring would be helpful"

You are right and i am working on it for months. It is just not easy to make. Especially if your team is small

I think the OpenAI left panel/ right panel code, seems to fit this.

This is what we are going for in Closer - an AI native experience that extends the mind of salespeople. https://closer.so

[flagged]

How about contextual AI insights as you're interacting with the world, wherever you are? Like a quality signal.

Oh wait, this is something FB/Goog could never do because what would it say about the quality of information hosted...

aren't we missing the point that co-pilot is there in case pilot gets incapacitated?

Love the idea & spitballing ways to generalize to coding..

Thought experiment: as you write code, an LLM generates tests for it & the IDE runs those tests as you type, showing which ones are passing & failing, updating in real time. Imagine 10-100 tests that take <1ms to run, being rerun with every keystroke, and the result being shown in a non-intrusive way.

The tests could appear in a separated panel next to your code, and pass/fail status in the gutter of that panel. As simple as red and green dots for tests that passed or failed in the last run.

The presence or absence and content of certain tests, plus their pass/fail state, tells you what the code you’re writing does from an outside perspective. Not seeing the LLM write a test you think you’ll need? Either your test generator prompt is wrong, or the code you’re writing doesn’t do the things you think they do!

Making it realtime helps you shape the code.

Or if you want to do traditional TDD, the tooling could be reversed so you write the tests and the LLM makes them pass as soon as you stop typing by writing the code.

This whole Copilot vs HUD debate instantly brought to mind a classic Japanese anime from 1991 called Future GPX Cyber Formula (https://en.wikipedia.org/wiki/Future_GPX_Cyber_Formula). Yeah, it’s a racing anime set in the then-distant future of 2015, where cars come with full-on intelligent AIs.

The main character’s car, Asurada, is basically a "Copilot" in every sense. It was designed by his dad to be more than just a tool, more like a partner that learns, adapts, and grows with the driver. Think emotional support plus tactical analysis with a synthetic voice.

Later in the series, his rival shows up driving a car that feels very much like a HUD concept. It's all about cold data, raw feedback, and zero bonding. Total opposite philosophy.

What’s wild is how accurately it captures the trade-offs we’re still talking about in 2025. If you’re into human-AI interaction or just want to see some shockingly ahead-of-its-time design thinking wrapped in early '90s cyber aesthetics, it’s absolutely worth a watch.

I'm very curious if a toggle would be useful that would display a heatmap of a source file showing how surprising each token is to the model. Red tokens are more likely to be errors, bad names, or wrong comments.

AI building complex visualisations for you on-the-fly seems like a great use-case.

For example, if you are debugging memory leaks in a specific code path, you could get AI to write a visualisation of all the memory allocations and frees under that code path to help you identify the problem. This opens up an interesting new direction where building visualisations to debug specific problems is probably becoming viable.

This idea reminds me of Jonathan Blow's recent talk at LambdaConf. In it, he shows a tool he made to visualise his programs in different ways to help with identifying potential problems. I could imagine AI being good at building these. The talk: https://youtu.be/IdpD5QIVOKQ?si=roTcCcHHMqCPzqSh&t=1108

Absolutely agree, and spellchecker is a great analogy.

I've recently been snoozing co-pilot for hours at a time in VS Code because it’s adding a ton of latency to my keystrokes. Instead, it turns out that `rust_analyzer` is actually all that I need. Go-to definition and hover-over give me exactly what the article describes: extra senses.

Rust is straightforward, but the tricky part may be figuring out what additional “senses” are helpful in each domain. In that way, it seems like adding value with AI comes full circle to being a software design problem.

ChatGPT and Claude are great as assistants for strategizing problems, but even the typeahead value seems to me negligible in a large enough project. My experience with them as "coding agents" is generally that they fail miserably or are regurgitating some existing code base on a well known problem. But they are great at helping config things and as teachers in (the Socratic sense) to help you get up-to-speed with some technical issue.

The heads-up display is the thesis for Tritium[1], going back to its founding. Lawyers' time and attention (like fighter pilots') is critical but they're still required in the cockpit. And there's some argument they always will be.

[1] https://news.ycombinator.com/item?id=44256765 ("an all-in-one drafting cockpit")

There’s a lot of ideation for coding HUDs in the comments, but ironically I think the core feature of most coding copilots is already best described as a HUD: tab completion.

And interestingly, that is indeed the feature I find most compelling from Cursor. I particularly love when I’m doing a small refactor, like changing a naming convention for a few variables, and after I make the first edit manually Cursor will jump in with tab suggestions for the rest.

To me, that fully encapsulates the definition of a HUD. It’s a delightful experience, and it’s also why I think anyone who pushes the exclusively-copilot oriented Claude Code as a superior replacement is just wrong.

I think one key reason HUDs haven’t taken off more broadly is the fundamental limitation of our current display medium - computer screens and mobile devices are terrible at providing ambient, peripheral information without being intrusive. When I launch an AI agent to fix a bug or handle a complex task, there’s this awkward wait time where it takes too long for me to sit there staring at the screen waiting for output, but it’s too short for me to disengage and do something else meaningful. A HUD approach would give me a much shorter feedback loop. I could see what the AI is doing in my peripheral vision and decide moment-to-moment whether to jump in and take over the coding myself, or let the agent continue while I work on something else. Instead of being locked into either “full attention on the agent” or “completely disengaged,” I’d have that ambient awareness that lets me dynamically choose my level of involvement. This makes me think VR/AR could be the killer application for AI HUDs. Spatial computing gives us the display paradigm where AI assistance can be truly ambient rather than demanding your full visual attention on a 2D screen. I picture that this would be especially helpful for help on more physical tasks, such as cooking, or fixing a bike.

About a decade back Bret Victor [1] talked about how his principle in life is to reduce the delay in feedback, and having faster iteration cycles not just helps in doing things (coding) better but also contributes to new creative insights. He had a bunch of examples built to showcase alternative ways of coding, which is very close to being HUDs - one example shown in the OP is very similar to the one he presents to "step through time to figure out the working of the code".

[1]: https://www.youtube.com/watch?v=PUv66718DII

We also need what goes along with HUDs : switches, nubs, switches, dials. Actual controls.

Although we are talking HUDs, I'm not really talking about UI widgets having the good old skew-morphism or better buttons. In the cockpit the pilot doesn't have his controls on a touch screen, he has an array of buttons and dials and switches all around him. It's these controls that are used in response to what the pilot sees on the HUD and it's these controls that change the aircraft according to the pilots will, which in turn change what the HUD shows.

I think there is a third and distinct model which is AI that runs in the background autonomously amd over a long period and pushes things to you.

It can detect situations intelligently, do the filtering, summarisation of what’s happening and possibly a recommendation.

This feels a lot more natural to me, especially in a business context when you want to monitor for 100 situations about thousands of customers.

The current paradigm is driven by two factors: one is the reliability of the models and that constraints how much autonomy you can give to an agent. Second is about chat as a medium which everyone went to because ChatGPT became a thing.

I see the value in HUDs, but only when you can be sure output is correct. If that number is only 80% or so, copilots work better so that humans in the loop can review and course correct - the pair programmer/worker. This is not to say we need ai to get to higher levels of correctness inherently, just that systems deployed need to do so before they display some information on HUD.

> anyone serious about designing for AI should consider non-copilot form factors that more directly extend the human mind.

Aren't auto-completes doing exactly this? It's not a co-pilot in the sense of a virtual human, but already more in the direction of a HUD.

Sure you can converse with LLMs but you can also clearly just send orders and they eagerly follow and auto-complete.

I think what the author might be trying to express in a quirky fashion, is that AI should work alongside us, looking in the same direction as we are, and not being opposite to us at the table, staring at each other's and arguing. We'll have true AI when they'll be doing our bidding, without any interaction from us.

Great post, informative and precise.

I think the challenge is primarily the context and intent.

The spellchecker knows my context easily, and there is a setting to choose from (American English, British English, etc.), as well as the paragraphs I'm writing. The intent is easy to recognise. While in a codebase, the context is longer and vaguer, the assistant would hardly know why I'm changing a function and how that impacts the rest of the codebase.

However, as the article mentions, it may not be a universal solution, but it's a perspective to consider when designing AI systems.

This is how ship's AI is depicted in The Expanse (TV series) and I think it's really compelling. Quiet and unobtrusive, but Alex can ask the Rocinante to plot a new course or display the tactical situation and it's fast, effective and effortlessly superhuman with no back-talk or unnecessary personality.

Compare another sci-fi depiction taken to the opposite extreme: Sirius Cybernetics products in the Hitchhikers Guide books. "Thank you for making a simple door very happy!"

Nobody mentioned Manna [1] yet? That suggests a mostly audio headset giving orders. There is a real-world version using AR glasses.[2]

[1] https://marshallbrain.com/manna1

[2] https://www.six-15.com/vision-picking

Great post! i've been thinking along similar lines about human-AI interfaces beyond the copilot paradigm. I see two major patterns emerging:

Orchestration platforms - Evolution of tools like n8n/Make into cybernetic process design systems where each node is an intelligent agent with its own optimization criteria. The key insight: treat processes as processes, not anthropomorphize LLMs as humans. Build walls around probabilistic systems to ensure deterministic outcomes where needed. This solves massive "communication problems"

Oracle systems - AI that holds entire organizations in working memory, understanding temporal context and extracting implicit knowledge from all communications. Not just storage but active synthesis. Imagine AI digesting every email/doc/meeting to build a living organizational consciousness that identifies patterns humans miss and generates strategic insights.

just explored more about it on my personal blog https://henriquegodoy.com/blog/stream-of-consciousness

My aha moment into this direction happened when i got XR glasses and tried embracing voice agents to see where this leads. Did not expect the HUD aspect be so cool of how this setup works. At first i tried using the glasses as a replacement for my main screen and using voice agents as a replacement to the sidebars and chat windows we are used to, which worked ok but then i started using the main screen and main agent interfaces again but kept the glasses with a second screen on a different visual focal plane connected to the voice agents. Dark mode gets a totally new importance there because black is transparent in XR which is important to work well without obscuring the main screen. I can switch between screens by shifting eye focus and have a sense what the other screen is doing a far better than just how a physical second screen is in the visual periphery. When i need to combine images more i move the focal planes closer together so i can see both screens better at the same time. The voice agents can answer side questions, start research that is needed for the next step of the main workflow or fix minor issues that are not important enough to interrupt the main workflow. Its so obvious how HUD, voice ai, XR and agents can grow together into this new computing environment but i am afraid what happens once android and ios shape that reality. I want this to be part of the web.

Been thinking about something similar, from fairly grounded ideas like letting a model autogenerate new features with their own name, keybind and icon, all the way to silly ideas, like letting a model synthesize arbitrary shader code and just letting it do whatever within the viewport. Think the entire UI being created on the fly specifically for the task you're working on, constantly evolving in mesh with your workflow habits. Now if only I went beyond being an idea man...

It's like the magic moment when GUIs replace CLIs.

I remember the first time I started up Win95 from DOS days. Stunning.

I think there's room for a mixture of both use-cases. I think personally programming is far more like being a mixture of a pilot and an engineer. There are times, like in this article where it's far more useful to have a HUD, while i'm focused, i'd much rather have a better eye on things that are interesting to me. I often find things like cursor do the opposite - they slow down your processing time, you shift into an environment that often feels like i'm correcting a junior engineers work rather than expanding my knowledge of the codebase.

For example, if i'm working on authentication logic, i'd much rather see a smart heads up display, you look at a function and get advice on where this might mess other things up in the codebase, edge cases, etc. A smarter form of current IDES that doesn't mean you click through 50 different files to work out that this third party package doesn't work on specific code. A HUD in this case is ideal.

But I find there's a more detailed, slower development, I still really often use the chat function on claude mixed with Obsidian to hold bits of information i've found useful, this is more related to getting a deeper understanding of certain concepts. As a stupid developer, I often find I might need something explained 20 times in 20 different ways, and actually a predictive text model is perfect in so many circumstances to explain a massive algorithm step by step. It's ideal for things like shader code, where I might come back to it 6 months later and want to work out what was in my head at the time, a historic chat is perfect for those things.

There's definitely a balance to be struck, I think now that the hype cycle is peaking we can hopefully seperate the profit seeking AI tools from the useful day to day knowledge expansion. Currently it feels like the discovery of perspective in the reneissance - instead of using it to further advancements, we're attempting to sell perspective courses to people.

I want a link to the GitHub for this[0] which he linked to. Makes Prolog quite interesting.

[0]: https://www.geoffreylitt.com/2024/12/22/making-programming-m...

Kind of a weird article because the computer systems that is "invisible" i.e. an integrated part of the flight control systems - is exactly what we have now. He's sort of arguing for .... computer software.

Like, we have HUDs - that's what a HUD is - it's a computer program.

I like HUDs as an analogy for other forms of content/experience enrichment too e.g. what's a HUD for other forms of interaction than just looking at screens (like audio, or multimodal).

Can we embed useful data behind the content we produce/consume or derive personalised versions for the user?

-> Text/audio/video tailored for me and my interests? (ie. not just content recommended for me, but the content itself is tailored for me in terms of the insights and practical applications) -> Podcasts I can interact with? -> Audiobooks I can ask questions about?

Richer interaction that allows for more back&forth and key insights for the user and use case.

I love this analogy.

I've been particularly feeling like this regarding AI code reviewers recently - I don't want a copilot that will do their own review, I want a hud that will make it easier for me to understand the change.

I've been toying with crafting such a code review tool as a side project recently: https://useglide.ai

You know that feature in JetBrains (and possibly other) IDEs that highlights non-errors, like code that could be optimized for speed or readability (inverting ifs, using LINQ instead of a foreach, and so on)? As far as I can tell, these are just heuristics, and it feels like the perfect place for an “AI HUD.”

I don’t use Copilot or other coding AIs directly in the IDE because, most of the time, they just get in the way. I mainly use ChatGPT as a more powerful search engine, and this feels like exactly the kind of IDE integration that would fit well with my workflow.

I would love to see a HUD that allows me to see the change that corresponds to Claude Code's TODO item.

I don't want inline comments as those accumulate, don't get cleaned up appropriately by the LLM.

This is now (more) dated. Copilots as an interface are dated. The current initiative is full agents with human in the loop at the very start, occasionally the middle and the end.

Huds are just good UI, something copilots can natively exist as part of in the form of contextual insights and alerts.

We’re moving on to agency where it’s everything else vs an entirely different entity taking the action of flying the plane from take off to landing.

100% agree with this. I want augmentation not automation.

That said, CoPilot in the form of "autocomplete" is kind of that.

I have been enjoying the hell out of Claude Code, but I'd feel much better about it if it wasn't a case of "here take this pile of diffs" and had a mode Socratic modality.

A tool that works with me instead of for me, because I'm going to have to review everything it does anyways.

I might be wrong but isn't the HUD the author suggesting for coding is basically AREPL? For debugging I can see it work, but chatbox and inline q&a I feel has awider application.

On a wider note, I buy the argument for alternative interfaces other than chat, but chat permeates our lives every day, smartphone is full of chat interfaces. HUD might be good for AR glasses though, literal HUD.

> design the cockpit so that the human pilot is naturally aware of their surroundings.

This is a design interface problem. Self-driving cars can easily ingest this HUD. This is the reason what makes Apple's AI different from other microservice-like AI. The spell checker, rewrite, proofread are naturally integrated into the UI to the extent it doesn't feel like AI powered operations.

I read you, but unfortunately despite the limitations of current AI tech, all companies covertly/overtly market themselves as human replacement not human augmentation solutions with the aim of cutting out expensive experts out of the loop, not making them more productive, even though the messaging might say otherwise from time to time.

I believe this is the framework vs library debate but in a different context. HUD is like me using a library of AI enabled tools. The human agency is given more importance here - they decide how to use the tools to suit their purpose.

Copilot is more like a framework where an AI system exists which tells me what to do (a bit like the inverse of a library).

Heads-Up-Display is of course useful if it gives you useful information. But you can't tell that display "Ok fly me to Rome". That's what "co-pilot" is about. It can do things, rather than just display information it decides you might find useful.

A very relevant article from lesswrong, titled Cyborgism https://www.lesswrong.com/s/f2YA4eGskeztcJsqT/p/bxt7uCiHam4Q...

> “You’ll no more run into another airplane than you would try to walk through a wall.”

People do walk into walls, though.

I see the appeal of the idea, but it's not a replacement for something that actually agressively demands your attention when there are high risk events.

I want AI (machine learning statistical techniques) to convert my brainwaves into text so that I can think my programs into the computer. That would be a useful breakthrough in AI-assisted programming,

I was recently thinking, if services offered sufficient api docs/specs you could really just provide an llm to the client and have any client build their own UI to fit their needs. Maybe not that you’d want to. But you could!

The author gives an example of a HUD: an AI that writes a debugger program to enable the human developer to more easily debug code.

But why should it be only the human developer who benefits? What if that debugger program becomes a tool that AI agents can use to more accurately resolve bugs?

Indeed, why can't any programming HUD be used by AI tools? If they benefit humans, wouldn't they benefit AI as well?

I think we'll be pretty quickly at the point where AI agents are more often than not autonomously taking care of business, and humans only need to know about that work at critical points (like when approvals are needed). Once we're there, the idea that this HUD concept should be only human-oriented breaks down.

HUDs are primarily of use to people that are used to parsing dense visual information. Wanting HUDs on a platform that promises to do your work for you is quite pointless.

Some sort of neural interface feels inevitable at this point

A thought-provoking analogy!

What comes immediately to mind for me is using embeddings to show closest matches to current cursor position on the right tab for fast jumping to related files.

needed to find a kids’ orthodontist. made a tiny voice agent: feed it numbers, it calls, asks about price/availability/insurance, logs the gist.

it kind of worked. the magic was the smallest UI around it:

- timeline of dials + retries

- "call me back" flags

- when it tried, who picked up

- short summaries with links to the raw transcript

once i could see the behavior, it stopped feeling spooky and started feeling useful.

so yeah, copilots are cool, but i want HUDs: quiet most of the time, glanceable, easy to interrupt, receipts for every action.

Actual HUDs, like the ones for cars - along with vector displays, and laser display tech in general, is criminally undervalued and underutilized.

I'm thinking AR glasses with AI HUDs and we're basically there - my to-do list just got a whole lot more interesting.

I'm pretty sure this is want I want to work on for the next decade if anyone's looking for a world class UI engineer.

Agree. Current copilot bullshits lost their context, ahhh no, they know nothing bout context, such a joke.

I do think typing text prompts at LLMs will go down as a barely remembered brief period in history, that seems quaint and dumb to anyone who remembers or is told about it.

(I don't know what we'll be doing instead, I just think text prompts feel dumb.)

One of the simplest and best working applications of AI is the gpts living in your clipboard. Any prompt/workflow you have is assignable to a shortcut on demand and it pops up a chat window with response on demand. It's been game changing honestly.

More sensors and more data, humans become a limiting factor until a better way to communicate the information is used.

Concurrent posting from 5 hours ago (currently no comments): https://news.ycombinator.com/item?id=44705018

makes sense, i guess tools like cluely are HUDs

Yes, this is a non-creepy way of applying AI.

I want an AI HUD that blocks ads.

It should have a paperclip mascot

I mean, fundamentally the reason that we don't build most AI HUD ideas right now is that they are either stupidly insanely expensive at scale, or you have to run the AI on-device on an NPU so underpowered that it's going to give you really really bad results.

Lots of great ideas in this space but it's tough to make something that delivers value and also is economically viable

So... Google Glass?

Enough AI copilots and AI HUDs, we need “none of this bullshit forced upon us”.

Let's stop allowing this clearly incentivized hype for investable shit and have honest conversation, please.

Excited for the next wave of ai innovation.

Ben Shneiderman wrote "Beyond Intelligent Machines" published in the Jan 1993 IEEE Software, and the subject of this hn discussion:

Beyond Intelligent Machines: Just Do It (1993) (umd.edu)

https://news.ycombinator.com/item?id=22742100

Mark Weiser and Ben Shneiderman and I worked at the University of Maryland College Park before he went to PARC, and they have a similar take on agents versus augmentation.

I think Douglass Engelbart, Maurice Wilkes, and Joseph Weizenbaum would agree with many of their points.

https://www.cs.umd.edu/~ben/papers/Shneiderman1993Beyond.pdf

Easier to read text version:

http://www.cs.umd.edu/hcil/trs/93-03/93-03.html

Beyond Intelligent Machines, IEEE Software, January 1993

https://news.ycombinator.com/item?id=22744573

DonHopkins on April 1, 2020 | parent | context | favorite | on: Beyond Intelligent Machines: Just Do It (1993)

I love this article! I worked with Ben Shneiderman at HCIL, and when he sent me a copy of this article to review, I was inspired by the "Dynamic Queries" of his "Dynamic Home Finder" demo, to implement something like that for SimCity (the Frob-O-Matic window). [Video and transcript below.]

Here are some of the main points of his article:

Don't label machines as "intelligent".

It limits the imagination. We should have greater ambitions.

Enable humans to accomplish tasks that weren't before possible, instead of trying to enable machines to accomplish tasks people can already do.

Predictability and control are desirable qualities. Give users the feeling of mastery, competence, and understanding, sense of accomplishment.

The "intelligent machine" label limits or even eliminates human responsibility.

If you treat machines like people, you're likely to end up treating people like machines.

Ben Shneiderman called his lab HCIL instead of CHIL, to explicitly put Humans before Computers.

Here's his description of "Dynamic Queries" and the "Dynamic Home Finder":

>Dynamic Queries. These animations let you rapidly adjust query parameters and immediately display updated result sets, which makes them very effective when a visual environment like a map, calendar, or schematic diagram is available. The immediate display of results lets users more easily develop intuitions, discover patterns, spot trends, find exceptions, and see anomalies.

>Figure 2 shows a screen from Dynamic HomeFinder, a prototype interface for real-estate agents that uses dynamic queries, written by Christopher Williamson of UM. Users can adjust the cost, number of bedrooms, and location of the A and B markers, among other characteristics, and points of light appear on a map to indicate a home that matches their specifications. Clicking on a point of light brings up a home description or image.

>Users of Dynamic HomeFinder can execute up to 100 queries per second (rather than one query per 100 seconds as is typical in a database query language), producing a revealing animated view of where high- or low-price homes are found-and there are no syntax errors.

>Our empirical study of 18 users showed Dynamic HomeFinder to be more effective than a natural-language interface using Q&A from Symantec (C. Williamson and B. Shneiderman, The Dynamic HomeFinder: Evaluating Dynamic Queries in a Real-Estate Information Exploration System," Proc. SIG Information Retrieval, ACM Press, 1992, pp. 338-346).

https://www.cs.umd.edu/users/ben/papers/Williamson1992dynami...

That inspired me to implement a version of the "Dynamic Home Finder" in SimCity:

Multi Player SimCityNet for X11 on Linux (Dynamic Home Finder / Frob-O-Matic demo at 3:35).

https://www.youtube.com/watch?v=_fVl4dGwUrA&t=3m35s

Transcript:

Here's an interesting thing inspired by Ben Shneiderman.

We can look at the ... This is the "Dynamic Zone Filter".

So we are going to set this to be dynamic.

Now it's going to show all the zones, but it's not going to show the ones that don't pass this filter.

So this filter is currently all the way open.

Now we're going to change population density.

This is a two-ended slider.

This is the segment of the population density, from zero to 81.

So everything else will disappear.

Say I'm looking for a home. I want low population density. But I want high rate of growth.

And now, these are the places that have low population density and high rate of growth.

And then you can just interactively ... So each of these filters out some of the places.

So you can look at ... I don't want high traffic density. I don't want any pollution. I don't want any crime. I want land value to be high.

I'm getting pretty picky. So maybe I'll deal with more people. Lower rate of growth. I'm too picky about pollution. I'm too picky about land value. That's it.

So basically, Ben Shneiderman demonstrated this as the "Dynamic Home Finder", and I realized that SimCity has all these layers of information that it can draw on, as fictitious as they are, to do that kind of real time, interactive, smooth database query.

It's just a much higher bandwidth way to query a database than is conventionally used.

Anyway, that was the dynamic zone finder.

Also: Here's some email I wrote to Ben about this article, after re-reading the article again in 2009:

Date: 28/02/2009 18:51 Subject: Re-reading "Beyond Intelligent Machines"

A long time ago, you sent us (me, Brad Myers, Jack Callahan and Mark Weiser) a preview of your IEEE Software article "Beyond Intelligent Machines", which I ran across and have re-read.

It's still delightful, inspiring and relevant today.

Speech synthesis and recognition has come a long way, to the point where the "talking car" scenario is quite common (TomToms that speak and recognize street names and addresses).

But I think the development of user-customizable user interfaces has been dreadfully stalled since HyperCard died.

An "Interactive Learning Environment" is a great description of what I'm developing SimCity into.

Remember when you visited CMU and I showed you the version of SimCity with the dynamic query feature, an "homage" to your Dynamic Home Finder? You could dial a series of filters on spatial properties like population density, pollution, traffic, land value, police coverage, etc. That code is now open source, and I'm redeveloping it!

It's now very easy to write Python code to dynamically query and filter the map and data layers, and visualize with transparent colored tiles and pixel overlays, depending on arbitrary Python functions over the state of the map and its data layers.

And it's also possible to script agents in Python!

Here's a picture of SimCity with the traffic overlay enabled (showing yellow/orange/red haze over high traffic areas), with some PacMan agents, which are programmed to follow roads, go towards high traffic and eat the cars! Notice the clear road behind each PacMan!

http://www.donhopkins.com/home/images/SimCityPacMan.png

    -Don

Micropolis Online (SimCity) Web Demo:

https://www.youtube.com/watch?v=8snnqQSI0GE

A demo of the open source Micropolis Online game (based on the original SimCity Classic source code from Maxis), running on a web server, written in C++ and Python, and displaying in a web browser, written in OpenLaszlo and JavaScript, running in the Flash player. Developed by Don Hopkins.

Source Code: https://github.com/SimHacker/micropolis

HAR 2009 talk: Constructionist Educational Open Source

https://medium.com/@donhopkins/har-2009-lightning-talk-trans...

Isn’t that what all the AI browsers like Comet, or the things like Cluey are trying to do

This is a mess.

The 1992 talk wasn't at all about AI and since then our phones have given us "ubiquitous computing" en masse.

The original talk required no 'artificial intelligence' for relevance which makes it strange to apply to todays artificial intelligence.

The original talk made good points for instance "voice recognition" has been solved forever at a reasonable level, yet people kept claiming if it was 'better' a 'magic experience' would pop out as if voice was different to typing. Idiots have been around for a long time.

Don't get what OP is trying to say.

'AI HUD metaphors' are very hard, that's why they are not ubiquitous, they require constant input. Spellcheck runs every character typed. Agents are because of less $.

'Hallucinations' also make 'AI HUD metaphors' problematic, for spellcheck squiggly red lines would be blinking on off all over the page as a LLM keeps coming back with different results.

The main reasons I don't have "smart" devices in my house:

1. They introduce new and fascinating failure modes that never happened with the old, dumb devices (e.g. if your router fails you lose the ability to control your lights).

2. They demand human attention at the slightest provocation (e.g. the microwave beeps loudly forever when your food is done, every app on your phone insists on interacting with you whenever the company would like to upsell you something, etc.)

Item 2 above is what TFA is about. Yes you can often turn this shit off, but that's not the point. The point is you shouldn't have to. Useful technology should never call attention to itself in the manner of someone with narcissistic personality disorder.

But what about emergency situations? Glad you asked. Many airplane crashes in modern aircraft have happened because of "warning buzzer overload" which happens when one important system on the aircraft fails and then causes a cascade of secondary warnings, while giving the pilot no insight as to the root cause. A true AI assistant would reason about such situations and guide the pilot toward the root solution.

A true coding assistant would do the same kind of reasoning about program errors and suppress multipage error dumps in favor of flagging the root issue.

I'd love to mount a top-down webcam above me when I work on electronics and have a real time feed into ChatGPT or similar for Q&A, for a real time coach that can see. One day... :)

doing AI code edits on a JS web app is probably the way to do it. probably using something like groq kimi k2 + morph for fast edits

now we need a web framework with faster reload times

AI powered this, AI powered that, what most people don't realise is that the next game changer would be Al powered batteries

> The agentic option is a “copilot” — a virtual human who you talk with to get help flying the plane. If you’re about to run into another plane it might yell at you “collision, go right and down!”

Planes do actually have this now. It seems to work okay:

https://en.m.wikipedia.org/wiki/Traffic_collision_avoidance_...