View Post

Bliss Chapman, Brain Interface Software Lead at Neuralink discusses UX with Lex Fridman

In Engineering, Experience Design, Medical Products, Science, Technology by Fredy Ore

Lex Fridman has interviewed the team at Neuralink and discuss technology, design & modelling challenges of creating a natural & intuitive brain-computer interface (BCI) experience. This podcast goes deep behind the scenes at Neuralink with discussions that goes back to the core of User Experience such as discussions on feedback loops, model integrations, probabilistic metrics, consistency actions, prompts & behavioral psychology.

Neuralink is a neurotechnology company founded by Elon Musk with the ambitious goal of developing advanced brain-computer interfaces (BCIs) to enhance human cognition and address neurological disorders. The company’s primary focus is on creating cutting-edge implantable devices that can seamlessly connect the human brain with external digital systems.

As UX, Design & many other disciplines merges with AI, LLM’s, new Tools and Computation Design, there is a clear shift to how we design, and the processes we use – a reflection to how our industry is shifting in only 2-3 years.

This incredible set of x5 interviews by Lex Fridman covers the ambitious steps of developing advanced BCI’s to enhance human cognition and achieve the Neuralink aim to address neurological disorders.

Although the podcast is very broad covering neuroscience, the human brain, surgery, implants, UI, cognition and technology, it was discussions on UX, Design and AI that was intriguing. The design for different human intentions & actions, interfaces and ways of reducing frictions through innovative technology and design was refreshing.

The Bliss Chapman interview, who is Neuralink’s Brain Interface Software Lead is a masterclass discussion on User Experience Design – the tip of the iceberg of topics today’s designers must focus on.

If you are interested in UX, Design and Technology, you will find what Bliss Chapman and Lex Fridman discuss incredibly eye-opening – especially how UX today merges with Machine learning Models, Control strategies, Closed vs. Open Feedback loops, Smoothing/Friction Biases and how Latency can be used as a cost metric in User Experience Design.

Bliss Chapman interview chapters:
5:06:11 – Bliss Chapman 5:19:27 – Neural signal 5:26:19 – Latency 5:30:59 – Neuralink app 5:35:40 – Intention vs action 5:46:54 – Calibration 5:56:26 – Webgrid 6:19:28 – Neural decoder 6:40:03 – Future improvements.

Below are some quotes:

05:49:28 – Bias Correction

On Bias Correction … and it turns out that that’s one of the first qualia of the cursor control experience that’s impacted by neuron. Quality of the cursor experience – It’s very poetic. I love it.

The quality of the cursor experience.

Yeah, I mean it sounds poetic, but it is deeply true. There is an experience. When it works well, it is a joyful… A really pleasant experience. And when it doesn’t work well, it’s a very frustrating experience. That’s actually the art of UX, you have the possibility to frustrate people, or the possibility to give them joy.

05:52:17 – Brain Laws of UX

on Brain Laws of UX … I think we’re in the early stages of discovering those laws, so I wouldn’t claim to have solved that problem yet, but there’s definitely some things we’ve learned that make it easier for the user to get stuff done. And it’s pretty straightforward when you verbalize it, but it takes a while to actually get to that point, when you’re in the process of debugging the stuff in the trenches.

05:53:14 – Assigning costs to every action (when an error occurs)

On using a cost metric … So there’s different sort of cost functions associated with errors in this space, and part of the UX design is understanding how to build a solution that is, when it’s wrong, still useful to the end user.

It’s so fascinating, assigning cost to every action when an error occurs. So every action, if an error occurs, has a certain cost, and incorporating that into how you interpret the intention, mapping it to the action is really important. I didn’t quite, until you said it, realize there’s a cost to sending the text early. It’s a very expensive cost.

06:10:10 – UX cannot just be solved iteratively, you need to step back & think globally

UX is not something that you can always solve by just constant iterating on different things. Sometimes you really need to step back and think globally, “Am I even in the right sort of minima to be chasing down for a solution?” There’s a lot of problems in which sort of fast iteration cycle is the predictor of how successful you’ll be… But UX is not that way, I mean, users are actually quite often wrong about what the right solution is, and it requires a deep understanding of the technical system, and what’s possible, combined with what the problem is you’re trying to solve. Not just how the user expressed it, but what the true underlying problem is to actually get to the right place.

The Full Transcript of the Lex Fridman Podcast #438.

View Post

Apple WWDC24 & updates to VisionOS 2 Spacial computing Experiences

In Artificial Intelligence, Design, Digital Experiences, Engineering by Fredy Ore

As Apple’s WWDC 24 kicks off this week following the Keynote announcement, Apple has started sharing some of the Developer conference events, with explanations on how to better design and develop for the upcoming changes to iOS 18, iPadOS 18, MacOS Sequoia, Apple Intelligence and VisionOS 2. There are also some interesting upcoming sessions I’m keeping a keen eye out for including updates to Wallet and Apple Pay.

I’ll try to recap the best videos from WWDC 24 below and also include those videos which further help support the education of Designers who are starting or transitioning into working closer with AI, Data, Visualization, Gestures or Development & Code.

VisionOS 2 brings new spatial computing experiences

Apple has introduced some new gestures to VisionOS which improves the navigation and access of the Home View. A simple rotation of the wrist now opens up the date and time, volume controls and battery levels to improve usability and orientation within Spacial environments and improvements to notifications or shared Mac displays within VisionOS.

Designing a multiview experience with Video Playback in VisionOS
VisionOS 2 introduces Context Controls for up to 5 videos and adjusts the UI based on use, focus & location placement.
Adopting zoom transitions in navigation to enhance UI animations and transitions with SwiftUI & UI Frameworks
Next Generation of the CarPlay design system

An area not covered in the keynote includes some of the enhancements to the CarPlay Design System with a helpful introduction to the CarPlay Design System by Ben Crick (from the Human Interface Design team) on Instrumentation design and a detailed technical view of the Architecture of CarPlay DS with Tanya Kancheva (from the Car Experience Engineering team).

These 2 talks explore how adaptive Design Systems can facilitate co-branding, expressive gauges, layouts, fonts, dynamic content and a more deeply customizable and adaptable experience, allowing Car brands to express their own design philosophy and create iconic tailored experiences.

If you work with Design Systems for Apps or Web, it is sometimes useful to also look at how design systems are being created / used / updated beyond phone / tablet / desktop uses which are used across other contexts, uses and products – such as within modern vehicle designed experiences.

The full WWDC Design videos are on the Apple Developer site, with some recently added to YouTube as Playlists.

A recap of key videos from WWDC23 are the best way to get an understanding of new design considerations within Mobile, Tablet, Desktop & Spacial Computing. I recommend watching the following:

WWDC24

WWDC23

If you are new to Spacial Design or starting to work on User Input design or gestures in iOS, iPadOS or VisionOS these videos are a must watch.

View Post

Rabbit introduces the R1, a small AI voice assistant computer

In Artificial Intelligence, Digital Experiences, Experience Design, Technology by Fredy Ore

Rabbit Inc a new AI startup, has introduced a small AI assistant computer called the R1 at this year’s CES 2024 costing approximately $199.

Created in collaboration with Teenage Engineering, the Swedish electronic company known for their synthesizer & sampler sequencer product (OP1), the new Rabbit R1 is not intended to replace smart phones, but act as a complementary device using voice interactions similar to Alexa and Siri.

Packed into a small size (approximately of a stacked set of post-its notes), the device aims to return responses faster than OpenAI’s ChatGPT and other modern LLM’s out there.

Rabbit Inc, an AI startup introduces the R1 a new pocket-sized voice-assisted device.

Instead of relying on a smartphone’s App, Jesse Lyu (Rabbit’s founder and CEO) says that you to just ask the R1 via a push-to-talk button and connect to a whole range of Apps and API’s through automated scripts called “rabbits”.

These scripts carry out new or learnt tasks independently via a simple interface, display and voiced by an assistant through Rabbit’s operating system, called Rabbit OS.

Rabbit’s trained model (known as Large Action Model or LAM) learns the “visual interface” of existing apps and API’s and learns how these work, such that it can play music from Spotify or order an Uber via voice command, essentially teaching the LAM model how an App works. 

The LAM learns what an App’s UI looks like, and uses these learning’s to create shortcuts to simplify the interface and experience of the R1 using voice. Lyu says, that the LAM can be used and applied to any app anywhere.

Related links:

View Post

Generative AI report by BCG looks at the strategies & roadmap for Financial Institutions

In Artificial Intelligence, Experience Strategy, Financial Services, Innovation by Fredy Ore

BCG has published a report that offers a simplified set of routes for financial institutions to navigate the recent 2023 trends of Artificial Intelligence and Generative AI (Gen AI).

To what can easily start as complex strategies for financial businesses to implement, this report synthesizes an approach towards a comprehensive process to a Generative AI roadmap that focuses on roles, skill-sets, and strategies for teams, design leaders and executives (within Financial Services), to successfully identify the transformative steps and necessary actions for making AI decisions in 2024+.

In summary, the report highlights:

  • A forward-looking approach to Artificial Intelligence and Gen AI and the importance of a vision necessary to align correct cultures and people strategies.
  • Identifies challenges with iterative approaches or tentative tweaks that try to unlock value.
  • How competitive advantage in Gen AI relies on true commitments to change and a path beyond the previous goals and horizons.
  • Integration into existing networks is key, but re-imagining traditional operations for Gen AI requires a complete transformation.

A shift is the extraction of value from existing AI processes, which tends to be based around predictive AI models that focus on:

  1. Behaviors
  2. Preferences
  3. Needs
  4. Risks

To new approaches which Gen AI introduces, which requires value extraction that AI models now focus on:

  1. Understanding
  2. Convergence
  3. Synthesis and
  4. Creation

It’s worth a read.

AI-Driven Value Opportunities in Banks

Source: BCG article – A Generative AI Roadmap for Financial Institutions (Nov 2023) – by Stiene Riemer, Michael Strauß, Ella Rabener, Jeanne Kwong Bickford, Pim Hilbers, Nipun Kalra, Aparna Kapoor, Julian King, Silvio Palumbo, Neil Pardasani, Marc Pauly, Kirsten Rulf, and Michael Widowitz

References (updated):

Some useful definitions:

  1. Artificial Intelligence (Classic Definition):
    Refers to the simulation of human intelligence in machines that is programmed to think and learn like humans. It encompasses various technologies and techniques that enable machines to analyze, understand, and respond in a way that traditionally required human intelligence.
  2. John McCarthy’s AI Definition:
    Coined by computer scientist John McCarthy in 1956, Artificial Intelligence is the science and engineering of making intelligent machines, especially intelligent computer programs. It involves creating algorithms and systems that can perform tasks that typically require human intelligence, such as problem-solving, speech recognition, planning, and learning.
  3. Modern Perspective:
    In the contemporary context, Artificial Intelligence involves the development of algorithms and models that enable machines to process vast amounts of data, recognize patterns, and make decisions without explicit programming. Machine Learning and Deep Learning are integral components of modern AI, allowing systems to improve their performance through experience.
  4. Narrow AI (Weak AI) vs. General AI (Strong AI):
    Narrow AI refers to AI systems designed and trained for a specific task, while General AI aims to create machines with the ability to perform any intellectual task that a human being can. Currently, most AI applications fall under the category of Narrow AI, focusing on specialized functions like image recognition, natural language processing, and game playing.
  5. Machine Learning:
    A subset of AI, Machine Learning involves the development of algorithms that allow computers to learn from data. Instead of being explicitly programmed, these algorithms use statistical patterns to improve their performance over time as they are exposed to more data.
  6. Deep Learning:
    Deep Learning is a type of Machine Learning that involves neural networks with multiple layers (deep neural networks). Inspired by the structure and function of the human brain, deep learning has proven to be highly effective in tasks such as image and speech recognition.
  7. Natural Language Processing (NLP):
    NLP is a branch of AI that focuses on the interaction between computers and human language. It enables machines to understand, interpret, and generate human-like text, making it essential for applications like virtual assistants and language translation.
  8. General AI (Strong AI):
    General AI, or Strong AI, is a theoretical concept in Artificial Intelligence that envisions machines possessing the ability to perform any intellectual task at a level equal to or beyond that of a human being. Unlike Narrow AI, which is designed for specific tasks, General AI would have the capacity to adapt and excel across a broad spectrum of activities, demonstrating true cognitive flexibility and understanding.
  9. Generative AI (Gen AI): Generative Artificial Intelligence refers to a subset of artificial intelligence that involves systems capable of producing new content, data, or artifacts. Unlike traditional AI models that focus on recognition and classification tasks, generative AI aims to create novel outputs, often in the form of images, text, or other types of data. This approach is frequently associated with techniques such as Generative Adversarial Networks (GANs) and variational auto-encoders, which enable machines to generate realistic and creative content by learning patterns and structures from large datasets. Generative AI holds promise in various fields, including art creation, content generation, and even the synthesis of realistic, but entirely artificial, data.
  10. Predictive AI: Predictive Artificial Intelligence refers to the use of advanced algorithms and models to analyze historical data, identify patterns, and make predictions about future events or outcomes. This subset of artificial intelligence is particularly focused on forecasting and anticipatory decision-making. Predictive AI systems leverage machine learning techniques to learn from past data, discern trends, and develop models that can be used to predict future trends, behaviors, or events. Common applications of predictive AI include sales forecasting, risk management, demand planning, and personalized recommendations. By understanding patterns in data, predictive AI empowers businesses and organizations to make informed decisions and proactively respond to potential future scenarios.
View Post

Brad Frost’s keynote from Hatch 2023 – ‘Is Atomic Design Dead?’

In Design, Digital Culture, Disciplines, Engineering, Product & Service Design by Fredy Ore

The Hatch 2023 Conference in Berlin shared some interesting discussions within Design, Research and Product and some of the conference videos have recently been made available on YouTube including the keynote from Brad Frost titled Is Atomic Design Dead? from October 6, 2023.

Brad Frost originally coined the term “Atomic Design” back in 2014 via his website & blog, and later in his seminal 2015 book of the same name. This term was during a time when Design Systems and collaborative design tools (Figma/Sketch/Miro) were becoming common discussions within UX and Product Design circles. Here is his original Atomic Design presentation from 2015 from the An-Event-Apart conference in Austin.

The other Hatch 2023 conference Berlin videos are also worth checking out.

View Post

UX Mag publishes article on the building blocks for Continuous Discovery in Design

In Design, Disciplines, Experience Design, Product & Service Design by Fredy Ore

For the last few years, there has been more and more discussions around Continuous Discovery. The 2021 book by Teresa Torres titled Continuous Discovery Habits placed a spotlight again on the topic, particularly within larger teams and orgs working in Product, UX and Design.

Yuliia Galytska has recently shared an interesting insight to the process, steps and methods in a great article on UX Mag that breaks down continuous discovery within Design and Product teams.

Well worth a read.

Last year Tarek Sadi, Director of UX and Product Design at Blinkist also discussed Continuous Discovery in a Hatch podcast episode – Designing for your Product Vision prior to Hatch 2023 in Berlin.

Further references to Terra Torres’ book Continuous Discovery Habits (2021): Discover Products that Create Customer Value and Business Value and the 2016 Opportunity Solution Tree article.

Further links & resources: (updated)

View Post

Adobe Design takes a look at the new(ish) design specialization – Growth Design

In Business Design, Design, Disciplines by Fredy Ore

As our industry matures and continues to evolve and grow, its not surprising to read about emerging disciplines within design. Adobe recently published an interesting article on the closely linked discipline area with product design called, Growth Design.

Verna Bhargava, a Senior Design Manager at Adobe says that Growth Design “expands on the human-centered problem solving process by using scientific methods to improve business methods”.

Although we have seen related specializations in the past few years, such as “Employee Experience” and “Business Design”, Growth Design focuses on purely scaling improvements. The article is an interesting one, as it focuses on the the close relationships with core Product Design, where user customer insights are used to conceptually solve design problems.

Whilst Growth Design also focuses on the observation and experimentation with customer behaviors, Adobe says it specifically focuses on Scaling and Improvements, and together (with Product Design), helps create even more meaningful experiences.

Adobe’s current Global Growth team focuses on these 4 main areas:

  • Optimization: Understanding drop-offs, bottlenecks, and closing gaps to reach performance goals. We lean heavily on data to identify and prioritize the bite-sized ways we can make the current experience more seamless.
  • Experimentation: Approaching known problems and opportunities in a current experience in new ways, powered by new customer insights, emerging patterns and behaviors, and market research.
  • Innovation: Identifying and building high level concepts for product or business transformation to push boundaries, disrupt current markets, and get ahead in new ones.
  • Impact: Socializing insights and knowledge back to our cross functional and core product partners to influence strategies and product building.
Leading Design – The emergence of the growth designer – a close-up look at this new(ish) design specialization

An interesting follow-up will again be similar to past discussions to how these newish emerging areas fall within Design, the vast complex dimensions of teams in the industry, and how these specializations fit within well established fields such as Service Design, Strategy, and User Research.

Definitely worth a read.

View Post

The role of design in shaping new fictions within Web3 and AI

In Brand, Design, Digital Culture, Digital Experiences by Fredy Ore

Caio Braga, Fabricio Teixeira & Luciano Infanti have written an excellent article that explores design in new fictions – the invisible social contracts that we all accept and subscribe to without even thinking.

The article looks at the role design plays in what Steward Brand once described as pace layering – a set of different types of relationship layers, hierarchies & dynamic systems in our world.

These layers (such as Fashion/Art, Commerce, Infrastructure, Governance, Culture & Nature) have a cohesive relationship within society & move at different paces that affect and inform one another. Brand described these relationships as components within a system and each having different change-rates & different scales of size – informing how our world works today, is structured and also organised.

Braga, Teixeira, & Infanti explore the visual representation of Fictions (the aesthetic influences such as symbols, typography, colour, brand, etc) & where old merge with new ones through recent cultural shifts in technologies such as AI, new monetary & financial tools & uses and most recently the introduction of Web3 into the lexicon.

Blockchains, smart contracts, cryptocurrency, virtual reality, the metaverse, and NFTs are very much starting to introduce a new set of visual language, metaphors, interfaces & contexts that is important to take a moment to observe, understand and contextualise.

Apple iPhone 14 Pro & iOS 16 – Dynamic Island UI (September 2022)

As old fictions start losing their place, space opens for new fictions to emerge in parallel. New narratives, new truths, and new worlds are being created.

It might seem that we have no control over this chaotic, contradictory, and overwhelming process. But we must remember that within this transformational period, there is space for new, positive changes. As designers, we have a small but important part to play by shaping symbols and its structures.

Our role & responsibilities as Designers is more important now than ever. ✋

The article is well worth a read.

View Post

Designing for Large Screens in Material Design 3

In Best Practice, Design, Digital Experiences by Fredy Ore

André Labonté (a Product Manager on Material Design) has presented a great talk at Google I/O 2022 on Designing for Large Screens, covering helpful principles, ergonomics to consider, Material Design 3 updates, adaptive Apps, and considerations for new devices including foldables.

The video goes beyond the Stretch to Fit vs. Optimizing for Large Screens discussions from 2020.

 

It’s also worth checking out Google’s updated Adaptive Design Guidelines for Material Design 3

View Post

Principles & Best Practices for Designing in Arabic (in 2022)

In Brand, Design, Digital by Fredy Ore

A really helpful WWDC 2022 video by Mohammed Samir (a Designer in the Apple Design team) on Designing for Arabic (in 2022).

The video covers a lot of already widely known UX/Design considerations, but goes into Cultural, Use & other important areas such as UI Directionality, Typography, Iconography & Numerals. There is around 660 million people that uses the Arabic script today, which makes it the third most written language in the world after Latin and Chinese.

Below are some important Principles from Apple’s Human Interface Guidelines on RTL (Right-to-Left) such as supporting languages like Arabic and Hebrew via reversing an interface as needed to match the reading direction of related content.

A basic Example Flow of accesing an App from within an App Store

RTL Principles

Align a paragraph based on its language, not on the current context. When the alignment of a paragraph — defined as three or more lines of text — doesn’t match its language, it can be difficult to read. For example, right-aligning a paragraph that consists of LTR text can make the beginning of each line difficult to see. To improve readability, continue aligning one- and two-line text blocks to match the reading direction of the current context, but align a paragraph to match its language.

Don’t reverse the order of numerals in a specific number. Regardless of the current language or the surrounding content, the digits in a specific number — such as “541,” a phone number, or a credit card number — always appear in the same order.

Flip controls that show progress from one value to another. Because people tend to view forward progress as moving in the same direction as the language they read, it makes sense to flip controls like sliders and progress indicators in the RTL context. When you do this, also be sure to reverse the positions of the accompanying glyphs or images that depict the beginning and ending values of the control.

Arabic version of the video – Design for Arabic · صمّم بالعربي

View Post

Remote Design Sprint Templates (Keynote, Miro, Figma)

In Design, Disciplines, Experience Design, Product & Service Design by Fredy Ore

Throughout the pandemic, a number of us have had to modify, change & completely replace established methods and tools for Design which was deemed normal or best practice just a few months prior.

Replacement to such methods hasn’t been easy. It has required taking tactical steps through experimentation to continue delivering similar or better outcomes as running a Design Sprint prior to the pandemic.

Throughout the last few months, and as London and many other cities continue returning to lockdowns and restrictions, I have been putting together a list & guide to some of the most useful and helpful posts, links, threads and articles (to adjustments required) across our processes & practices to ensure we save time, avoid unexpected obstacles and deliver positive outcomes.

The first of these guides will be dedicated to the Design Sprint. I have prepared, facilitated and led a few through the years across projects with clients.

Jake Knapp, John Zeratsky & Jackie Colburn’s post is incredibly useful, as it addresses the biggest questions most are asking, such as How do you handle whiteboards, voting, facilitation, interviews, video and pretty much everything else when running a remote Design Sprint via Zoom.

The steps are very similar, but as Jake, John & Jackie mention, it is not perfect. It requires flexibility & adjustments to cater for your project, your situation and required outcomes from the Sprint.

Sarah McIlwain, Product Design Manager at Abstract recently wrote about running a remote Design Sprint, including benefits, tips and adjustments required to keep teams engaged for the 4 days.

Alongside this, the Design team at Spotify have also put together a Figma template for remote Design Sprints which saves a lot of time.

Created by Alexandria Goree the Figma sample template can be reused in Presentation mode as alternative Keynote versions for the activities on each day.

Both worth a look.

View Post

Upcoming iPadOS 15 introduces much needed Multi-tasking & new Translate features

In Artificial Intelligence, Design, Digital Experiences, Experience Design by Fredy Ore

Apple’s online WWDC Keynote 2021 is introducing some new features for iPad OS 15, including privacy features (private relay) and live translation including from your Photo library.

Pretty neat.

Notes is also being updated to include #Tags and @Mentions.

Translate can now also live translate and text in photos.

View Post

V&A 2021 re-opening day & the Alice Curiouser & Curiouser Exhibition

In Book, Design, Digital Experiences, Gallery by Fredy Ore

I had the wonderful pleasure of visiting the V&A (Victoria and Albert) museum for re-opening day on the 19th of May 2021 following the UK lockdowns and restrictions. It was such a wonderful experience returning to some normality, and visiting galleries again after a year or so of restrictions to public spaces in London & the UK.

The new Alice: Curiouser and Curiouser exhibition, which explores the origin, adaptations and reinventions of Lewis Carroll‘s book Alice’s Adventures in Wonderland is quite simply amazing.

From manuscript to a global phenomenon the exhibition explores over 157 years of original work, drawings, artwork, costumes, a VR experience and the modern Disney films. I completely recommend it even if you have younger children.


View Post

Config 2021 & Keynote videos are now on YouTube

In Design, Digital Experiences, Experience Design, Product & Service Design by Fredy Ore

Config 2021 was inspiring & a treat this year – particularly facing the challenges we all have working remotely due to the global pandemic. If you are new or already using Figma and collaborating remotely, the conference was for you – covering 2 days of inspiring talks by teams pushing the limits of Figma for use in Design Systems, collaboration within teams and more.


There were so many things I learnt this year, including loads of time-saving practical shortcuts and tricks I now apply every day. The conference was broken into 3 themes:

  1. Embracing the mess
  2. Shifting team culture
  3. In the file

If you missed it, all the recorded videos of Keynotes & talks are now live on Youtube.

Check them out.

These were my highlights:

View Post

Atomic Habits by James Clear

In Books, Health & Well-being by Fredy Ore

Over the last few months during lockdown, I’ve slowly been reaching out to a handful of books I’ve ignored for way too long in my reading list. One I have found extremely valuable has been the brilliant book by James Clear, Atomic Habits.

Well worth a read.

James Clear is an author and speaker focused on habits, decision-making, and continuous improvement. And has spoken at a few events. Below is a nice clip where he discusses the 4 stages of habit formation – 1.Noticing, 2.Wanting, 3.Doing, 4.Linking