
SmartTaste AI

Project name - SmartTaste AI
Role - AI UX Designer, AI Engineer
AI Tools - ChatGPT, Figma Make , Stitch, Google AI Studio, Gemini, Cursor, Xcode
Note- The brand name DrinklyAI has been updated to SmartTaste AI.
Project date - Oct, 2025
The information in this case study is my own and does not necessarily reflect the views of SmartTaste AI.
My role
I functioned as both the lead UX designer and developer to build an AI-powered nutrition platform. I managed the full design stack—research, synthesis, low-to-mid fidelity wireframing, interactive prototyping, and develop the application using AI.
AI-Augmented Development: Rather than following traditional, slow development cycles, I utilized an AI-first workflow. By integrating tools like ChatGPT, Figma, Stitch, Google AI Studio and Cursor, I accelerated the UI development and coding process in Android studio and Xcode. This allowed me to focus on 'Human-AI Interaction' (HAII) principles, ensuring that the AI components felt seamless and accessible to the end user.
_edited.png)
Project goal
The SmartTaste AI project serves as a blueprint for the future of AI-driven product development. From initial concept to cross-platform deployment on Android and iOS, every stage was optimized using a diverse AI tool stack.
Our goal was to demonstrate how AI can drastically reduce time-to-market without compromising UX quality. By launching a fully functional MVP (Minimum Viable Product) ahead of schedule, we established a continuous feedback loop with early adopters. This strategy focuses on rapid redesign and data-informed enhancements to build a robust, human-centered app capable of scaling to a global audience.
Overview
SmartTaste AI is an intelligent personal nutritionist application designed to revolutionize how individuals interact with their daily nutrition. Powered by advanced AI, the platform provides deep nutritional insights and hyper-personalized meal planning to help users make smarter, data-driven dietary choices.
Core Features & Capabilities
-
Hyper-Personalized Meal Planning: Generates dynamic daily food plans tailored strictly to individual user goals.
-
Deep Ingredient & Flavor Analysis: Utilizes advanced AI to break down complex ingredient lists, nutritional details, and flavor profiles.
-
Frictionless Health Tracking: Seamlessly monitors daily calorie intake and macro/micronutrient distribution.
-
Ask anything: Delivers responses about any questions on nutrition.
-
Cross-Platform Accessibility: Built and optimized for native experiences on both iOS and Android.
The design process
-
Research
- Data collection
- Survey
- User stories
-
Define
-
Ideate
-
Prototyping
-
Usability test
-
Development
Getting to know the business goal
First, I held a kickoff meeting with the business owners and product stakeholders to gain a clear understanding of the product vision, business goals, and overall requirements for the SmartTaste AI application. This meeting helped me understand both the user needs and the business objectives behind the project.
The stakeholders wanted to design and deploy both iOS and Android applications targeted at users who are looking to adopt a healthier lifestyle. During the discussions, I gathered valuable insights into the business strategy, product expectations, and future vision for the platform.
The primary business goal was to rapidly design and develop an MVP so real users could begin testing the application as quickly as possible. The long-term vision of SmartTaste AI is to create a personalized nutrition app that helps millions of users better understand their food and drink choices, encouraging healthier habits and smarter daily decisions.
The app aims to provide users with intelligent recommendations and insights that support healthier eating and drinking behaviors through an accessible and user-friendly mobile experience.
Target audience
SmartTaste AI is designed for a diverse demographic of health-conscious users, ranging from ages 13 to 65. Our platform prioritizes accessibility, catering to anyone with a smartphone who seeks a seamless way to track nutrition and improve their dietary habits.
Inclusive Design at the Core: We specifically engineered the app to support multiple interaction models:
-
Visual Learners: High-speed photo and barcode scanning for instant analysis.
-
Accessibility-First: Robust voice-command functionality designed for users with visual impairments.
-
Language: Currently 6 languages are supported
-
Precision Users: Text-based input for those who prefer detailed, manual logging and conversational AI interaction.
Since SmartTaste AI Involves with the daily life of may people I have maintain a great relationship with the user, our challenge was to evolve with customers and enter the highly competitive on‐demand food and drink digital industry.
SmartTaste AI offers a daily meal plan for benefits for premium subscriber members. With this benefit we hope to create deeper relationships with SmartTaste AI customers. Smart taste ai is always open for user feedback and customer suggestions.
Key Competitors
Cal AI, Yuka, MyFitnessPal, Vivino
Competitor analysis
I conducted a detailed competitive analysis using available AI tools and market research to better understand the strengths, weaknesses, and opportunities within the AI-driven nutrition industry. The goal was to identify gaps in existing solutions and uncover opportunities to create a more valuable experience for SmartTaste AI users.
As part of the research process, I analyzed leading applications such as Cal AI, Yuka, and MyFitnessPal. Each platform was evaluated based on features, user experience, AI capabilities, accessibility, and overall product positioning.
The research revealed that the AI nutrition market is rapidly shifting from manual food logging toward AI-powered, vision-first tracking experiences. Cal AI primarily focuses on fast calorie counting for fitness-oriented users, while Yuka emphasizes product and ingredient scanning. MyFitnessPal offers broader calorie and fitness tracking features with a strong focus on manual input and health monitoring.
Based on these insights, SmartTaste AI was positioned as a more holistic wellness companion with a broader and more user-centered feature set. The product strategy focused on combining AI-powered nutrition insights with additional features such as grocery planning, personalized recommendations, and multi-language accessibility to better support a diverse global audience.
Interview process
To prepare for the SmartTaste AI user research phase, I begin by establishing clear objectives centered around understanding how people manage their dietary habits and holistic health. Preparation involves crafting a semi-structured interview guide with open-ended questions designed to uncover authentic behaviors, frustrations with current tracking tools, and how users naturally define "healthy vs. unhealthy" options. To ensure a comprehensive understanding, I recruits a diverse group of eight target users, ensuring the qualitative insights gathered represent a broad spectrum of lifestyle goals and tech-savviness before any design work begins.
The execution of these sessions is orchestrated entirely around the users' convenience to foster comfort and openness. Utilizing remote collaboration tools, I conduct one-to-one Zoom interviews scheduled flexibly around the participants' availability. To optimize data collection without compromising human connection, the designer collaborates with a co-worker who serves as a dedicated note-taker. This division of labor allows me to focus entirely on engaging the user and probing deeper into their experiences, while the teammate handles session recording, tracks active documentation, and notes non-verbal cues in real time.
During the session, the interview transitions into a tactical usability test and contextual inquiry to observe how users interact with early concept ideas or competitive interfaces. This qualitative feedback is then systematically paired with quantitative analytics to paint a complete picture of the user landscape. While the one-to-one interviews reveal the emotional why behind user frustrations—such as navigating complex grocery lists or wanting specific language support—the numerical data validates the statistical significance of these patterns, ensuring the final feature set is grounded in proven user necessity.
Ultimately, this rigorous synthesis directly shapes the application's information architecture and visual hierarchy.
This empirical foundation guarantees that the upcoming design phase solves genuine user pain points with a highly tailored, intuitive interface.
Interview questions
SmartTaste AI: User Interview Guide
1. Introduction & Device Setup (Warm-up)
Goal: Build rapport, understand their tech ecosystem, and set expectations.
-
"To start off, what smartphone do you currently use as your primary device? (iOS or Android?)"
-
"In general, how often do you use your phone throughout the day, and what are the 2 or 3 apps you open the most?"
-
Follow-up: "How often do you use your smartphone specifically for cooking or meal preparation purposes? (e.g., looking up recipes, watching cooking videos, tracking groceries)."
2. Dietary & Hydration Habits (Behavioral Patterns)
Goal: Understand the user’s daily routine, baseline behaviors, and relationships with food/drinks.
-
"Walk me through a typical day for you: How often do you eat and drink in a single day, and what types of food and drink do you typically consume?"
-
Follow-up: "How do you currently decide what is 'healthy' versus 'unhealthy' when you buy groceries or order out?"
-
Follow-up: "How much attention do you pay to the liquid calories or ingredients in the beverages you drink throughout the day?"
3. Pain Points, Challenges, & Desired Changes (The "Why")
Goal: Uncover core frustrations, motivations, and areas where the user actively wants to improve.
-
"If you look at your current eating and drinking habits, do you face any specific challenges or frustrations?"
-
"Is there anything about your current diet or nutrition that you actively want to change?"
-
Follow-up: "What specific changes or improvements do you need to see in your eating and drinking habits to feel like you are living a healthier lifestyle?"
-
Follow-up: "What has stopped or blocked you from making those changes in the past?"
4. Competitive Experience & Feature Deep-Dive (Mental Models)
Goal: Gauge familiarity with existing tracking solutions and test reactions to SmartTaste AI’s core pillars.
-
"Have you ever used an application that tells you the details of your food and drink, or counts your calories? If yes, which one, and what was your experience like?"
-
Follow-up: "What did you like about it, and what made you stop using it or feel frustrated by it?"
-
Follow-up: "If you could simply take a photo of your food or drink to instantly get nutritional tips and exercise equivalents, how would that fit into your daily routine?"
-
Follow-up: "How important is it for you to have an app like this available in a specific language other than English (for example, Amharic or others) to share with family or use comfortably?"
Understanding the Problem
After gathering detailed insights from the business stakeholders, I analyzed the available data, existing documentation, and market research to fully understand the business objectives and identify the most effective solution for the problem.
One of the key challenges was the requirement to deliver the application within a very short timeframe while still creating a high-quality user experience. Conducting competitor analysis provided valuable insight into current market standards, user expectations, and gaps in existing products.
To better understand user behavior and pain points, I conducted usability testing and interviews with 15 participants, supported by analytics and quantitative data analysis. This research helped uncover several important user needs and frustrations.
The findings showed that users value simplicity and ease of use above all else. Many users preferred immediate access to the application without lengthy sign-up processes or unnecessary onboarding steps. Additionally, accessibility emerged as an important opportunity area, particularly for users with disabilities who lacked options to listen to or interact with food and drink information through audio support.
By combining business requirements with user research insights, I gained a clear understanding of both the user pain points and the business goals. This allowed me to define a solution strategy focused on speed, accessibility, simplicity, and user-centered design while ensuring the product could be delivered efficiently within the required timeline.
Challenge
The primary challenge of this project lay in balancing rapid execution with the high precision demanded by an intelligent system. Tasked with designing and deploying a high-traffic application at speed by leveraging advanced AI tools and a new design library, the immediate architectural decisions heavily impacted the final customer experience. The core UX and technical hurdle was achieving extreme accuracy and perfection within the AI's output; this required systematically testing and validating the nutritional results of various foods, drinks, and products against established competitor standards to ensure reliability. Furthermore, maintaining a premium, cohesive design and seamless user flow became a moving target due to continuous feature updates, which forced us to iteratively uncover true user pain points to ensure the AI integration actively drove long-term engagement rather than just novelty.
Strategy and Plan
To anchor the project from day one, I established comprehensive frameworks, UX guidelines, and a product strategy that clearly defined our core vision, design principles, content strategy, and development methodologies. A primary focus of this strategy was creating a clear roadmap for translating qualitative research findings—such as user pain points regarding complex nutrition data—directly into functional, intuitive application features. By establishing these frameworks early on, I was able to successfully evangelize design ideas, gain cross-functional alignment, and drive critical product decisions across engineering and stakeholder teams.
Artificial Intelligence serves as the foundational pillar of the SmartTaste AI experience. By leveraging cutting-edge AI technologies, models, and methodologies, we shifted the user paradigm from tedious manual input to intelligent automation. The overarching strategy focused on utilizing advanced computer vision for instant food and drink scanning, paired with large language models to deliver personalized, natural-language dietary coaching. This technical strategy ensured that the AI integration wasn't just a feature, but the core engine driving the application's seamless, minimal UX.
Define
Following an intensive phase of qualitative and quantitative research, I transitioned into the Define stage to synthesize our findings into actionable design requirements. By mapping the data gathered from our eight user interviews, usability testing, and numerical data analysis, I pinpointed exactly where user friction existed and what core challenges our audience faced. This careful analysis allowed me to bridge the gap between user pain points—such as the need for accurate, real-time nutritional scanning and holistic guidance—and a highly optimized product strategy.
With a clear problem definition in place, I mapped out the application's overall structure and engineered a robust Information Architecture (IA) to define exactly how users would intuitively navigate and interact with the product. To push the boundaries of what the AI technology could deliver, I then facilitated a "Crazy Eights" ideation session, generating a wide range of rapid, innovative concepts. This gave the team multiple layout and workflow alternatives to critique, compare, and refine, ensuring we collectively landed on the most seamless, high-engagement solution before moving into high-fidelity design.
Sketching
Sketching and Paper Prototype
Once the problem space and strategic direction were defined, I began the ideation phase with rapid, low-fidelity sketching. Leveraging my background in fine art and drawing, I explored a wide range of layouts and concepts on paper to map out the application's user flow and information architecture. This creative exploration culminated in a paper prototype of the strongest concept, allowing me to align the design direction with our brand identity, user needs, and business goals before moving into digital wireframes.









User Persona

Empathy map
Deliver Fast
In favor of speed to market, leveraging AI powered tools and applications we were tasked to design and build within the new design Library. with the help of AI I will delivered the design and full production application as quickly as possible
The assumption was simple—millions of customers will visit everyday.
The quick and early architectural decision had a major impact on the quality of the customer experience we did design and deliver fast. As always quick design and development to test in the hands of users.
Style guide
Here is the style guide for the visual design system, the typography, color, buttons, icons, interactive state.
The actual style guide hsa more features but i compose this for this presentation purpose.
also I have been aware of the accessibility guidelines and use them in grouping related contents, adding more color contrast, easy to spot text, adding visible indicators on icons and buttons and using different interactive states.
Ideas to action
I sketched countless ideas and brainstormed various possibilities with my design team and created low-fidelity wireframes and prototypes to test. Even though we took some important aspects from the first design, we came up with the second prototype which is similar but have much more changes, with clear, simple but most importantly meets the experience of our users. I invest significant time in meticulous prompt preparation, which is essential for ensuring seamless implementation, preventing duplication errors, and minimizing post-creation editing time.
SmartTaste AI offers a daily meal plan for benefits for premium subscriber members. With this benefit we hope to create deeper relationships with SmartTaste AI customers. Smart taste ai is always open for user feedback and customer suggestions.
Prompting while using AI
I use a clear, practical, and industry-level guide on how to prompt and structure my workflow when developing an DrinklyAI app while using tools like Figma AI, Google AI Studio, OpenAI Chat GPT, Gemini, Cursor. Here is the guideline I use while I prepare my prompt.
1. Complete App Overview & Goal
Define what the app does, who it serves, and what problems it solves.
Prompt focus: clarity of purpose.
2. Define the Architecture
Break the product into modules (e.g., onboarding, scanning, generating, matching).
Prompt focus: each module has its own role + constraints.
3. Create the Design System
Establish tone, voice, colors, UI behaviors, accessibility rules.
Prompt focus: make the model follow consistent language and UI logic.
4. Build a Prompt Template Library (per screen)
For each screen or feature, create reusable prompts.
Prompt focus: structure prompts using role + context + task + output format.
5. Continuously Evaluate Outputs
Test prompts with different scenarios and refine them.
Prompt focus: improve accuracy, clarity, and consistency.
6. Create Global Guidelines for App Behavior
Define how AI should respond across the entire app:
-
tone
-
constraints
-
formatting
-
allowed & not allowed behaviors
Prompt focus: global rules ensure predictable results.
7. Execution Order
Implement prompts in the correct flow:
System prompt
Developer instructions
User prompt
Output formatting
This creates stability and production-ready responses to my workflow.





Information Archtecture



.png)


Account creation user flow
Recognizing that account creation is the most critical touchpoint in the application, I prioritized a seamless, intuitive user experience. I designed an interactive mockup built on a 'clean path' architecture and a streamlined user flow. This approach ensures users can register effortlessly, eliminating the common friction points that previously led to abandoned sign-ups.











User Testing
Usability Testing & Iterative Refinement
Placing an interactive prototype into the hands of real users is the most effective way to validate what works and identify areas for improvement early in the design cycle.
I conducted moderated usability testing sessions with 10 participants via Zoom. During these sessions, I closely observed how users interacted with the interface, listened to their real-time feedback, and documented their behavior. This phase was crucial for gaining deep user empathy and uncovering critical friction points.
Our design process was entirely iterative. By synthesizing user feedback and mapping key pain points, we transformed user frustrations into actionable design solutions. Every major update—from core functionality to new features—was directly driven by participant insights, ensuring the application evolved around real-world needs. All interview data, recordings, and insights were systematically organized into a centralized repository for continuous reference.
Key Design Updates Based on User Feedback
The insights gathered from testing were immense and directly shaped the final pre-production design:
-
Gamification & Motivation: Added a Badge Section directly above the daily meal plan to reward healthy habits and increase user retention.
-
Onboarding & Personalization: Introduced Informational Onboarding Screens and Multi-Language Support to improve accessibility and ease users into the ecosystem.
-
Meal Planning Depth: Integrated a One-Week Grocery List within the daily food and drink tracker, alongside detailed Preparation Methods and Cook Times for all daily meals.
-
Optimized Logging Flow: Originally, scanned items were automatically sent to the "Recently Added" section. To give users more control, I updated this flow with a Dual-Action Prompt:
-
Option 1: Add to the "Recently Added" section to automatically log and deduct the calories from their daily goal.
-
Option 2: Tap "Done" to dismiss the item without logging it.
-
-
AI Accuracy Control: Added a dedicated feature allowing users to manually correct AI scan results if the automated nutritional data is inaccurate.
-
Smart Alternatives: Direct feedback regarding dietary frustrations led to the inclusion of an AI-driven alternative engine, which suggests healthier food and drink options whenever a user scans a poorly rated item.









Prompt
You are an expert full-stack developer and UX engineer. Build a responsive AI-
powered MOBIE application called SmartTaste AI, designed for IOS and Android
mobile app. For this project only focus on IOS AND MAKE SURE TO
FOLLOW IOS APPLE USSER INTERFACE GUIDELINES.
(Languages of our app to select from: English, Spanish, Chinese, French, German,
japan, Portuguese, Korean, Russian, Arabic,Amharic, Italian, Indian )
(One thing to remind is that we have to avoid nutritional advice for health or
medical users)
Application Structure:
The application will include screens:
-Splash screen,
-Sign up,
-Sign in,
-On boarding screens (consists of seven screens per question),
-Acknowledgement screen,
- Congratulations screen, You are getting there,
- Try SmartTaste AI ,Start your 3 day FREE trail to continue,
-Payment,
-Dashboard,
-My Profile
Design Instruction:
Use the uploaded design file as the visual reference. Follow consistent spacing, clean
alignment, responsive layout, and smooth navigation transitions.
Analyze images (the design) for overall style, color palette, and component design to
ensure consistency across all screens.
The page should be clean, professional, and minimalist with ample white space and
balanced padding for readability.
Use SmartTaste AIlogo for the app icon and splash screen use the uploaded logo, logo
color Black and white.
Main buttons- Primary Button color and all dark colors and text use our primary color
#3D3F76- and Sub text Use #8283A8- For Primary buttons use color-#3D3F76 Card
background color and secondary button -smaller - use #F3F1FF.
Cards used in this application- will have corner radius of 16.
Maintain a clean, trustworthy aesthetic suitable for a professional AI-powered
calorie tacker application.
Note- I will upload images so you can use the inspiration from the screens the
screens are for food if you have to use anything from the screens make sure to
update it to drinks.(since our app focuses on drinks)
Goal:
SmartTaste AI is an AI technology with smart way to understand our drinks better and
know our drinks smarter with the power of AI
To create a great application powered by AI that tracks calorie in drinks and detect
what it consist of and generates recommendations based on their personality and
preferences. create a solution using SmartTaste AI to solve the difficult problem of
volume and ingredient detection with high accuracy.
We focus on drinks this is our niche — also we have to solve the technical difficulty
of recognizing drinks + estimating hidden calories (sugar, alcohol, mix).
Combine image recognition (to detect container shape, color) + OCR (for reading
labels) + voice / text input to disambiguate.
Let users correct / confirm: after the app guesses what the drink is, you ask “Did you
add sugar / milk / flavoring?” to improve accuracy.
The main issue with liquid calories is that they often fail to trigger the same powerful
satiety (fullness) response as solid food. For a drink like a custom coffee or a cocktail,
it's nearly impossible for a user to guess the macros:
Our AI app that can detect these ingredients/volumes and provide a near-perfect
macro breakdown solves this core accuracy issue for the user.We have to track
calories or ingredients in mixed drinks, which is our core feature.
For Aicholic drinks we have to track on alcohol units, and the total caloric and sugar
breakdown of a complex cocktail or mixed drink. our strength should be accuracy and
detailed nutrient tracking, which is what our AI would need to deliver. focusing on AI
photo, text and voice inputs.
Our differentiating features should be:
"Photo Scan" ‘’Text’’ or ‘’Voice Record’’ Accuracy: The ability to accurately estimate
liquid volume (and therefore calories) in a non-standard glass from a single photo.
Ingredient Disaggregation: Identifying the components of a complex, mixed liquid
(e.g., espresso shot, steamed milk type, syrup flavor/pump count) and providing a
breakdown.
Real-Time Recommendations: Giving instant, highly contextual feedback like: "That
smoothie is 650 calories. Tap here to log a 350-calorie version with protein powder
and unsweetened almond milk."
The app will have a 3 day free trial and in app purchase with $29.99 per year
($2.49/mo)
Our app AI app should not just track the macros; it should provide the context that
makes macro counting for drinks successful: Macro Breakdown: Show the Protein,
Carb, and Fat grams clearly. Highlight Added Sugar: Make the refined carbohydrate
(sugar) count highly visible, as this is the primary villain in many drinks.Offer a
"Macro Swap" Feature: For high-carb/high-fat drinks, recommend a replacement that
preserves the protein or lowers the total caloric load (e.g., "Swap for a $10 \text{ g
Protein}$ $200 \text{ cal}$ version").
Alchol: In SmartTaste AI if a user scans for an alcoholic drinks in a popup ask ‘Hi is this
is an alcoholic drink? Please confirm.’ with Yes or No button and if no ask them
‘Thank you for confirming so what is this dink?’ show two buttons microphone and
pencil icons and user can text or write. confirm and continue. If yes and user is under
21 years of age at the time of scan do not show results of the photo, text or voice, let
the user know that ‘You are not 21 and older, I am not able to provide this for you
thank you.’’ And just a close button. also remove the drink they have scanned.
The application will have a Light and dark mood option on the splash screen and on
the dashboard- so users can have both options.
Depending on the user input during on boarding show all results of any amounts by
Lb or Kg.
Visual Solutions and Mockups
Visual Design & Iteration
After establishing the low-fidelity wireframes, I utilized AI to rapidly generate and iterate on the visual design. This allowed the team to explore multiple design directions quickly and efficiently.
1. Improving Feature Discoverability
-
The Problem: In the initial ideation stage, the Plus (+) icon was buried deep within the user flow and originally slated for its own dedicated page, severely limiting its discoverability.
-
The Solution: To solve this, we extracted the icon and placed it directly into the primary navigation menu.
-
The Outcome: Post-implementation data showed that users frequently utilized the prominent Plus icon to seamlessly navigate to other areas of the app. While the team was initially divided on this approach, the positive user engagement proved it was the right experience to move forward with.
2. Streamlining the Landing Page
I simplified the landing page to give the interface more "room to breathe" and drastically improve the visual hierarchy:
-
Enhanced Contrast: Swapped in a darker background image to ensure the main text fields and inputs stand out sharply.
-
Core Actions: Made the "Return" and "One-Way" toggle options instantly recognizable.
-
Call to Action: Enlarged the primary button and refined the microcopy for better clarity.
3. Optimizing Navigation & Accessibility
-
Navigation Swap: Replaced the "Settings" icon in the bottom navigation bar with "Profile," as user data indicated settings were rarely accessed, whereas profile management was a high-frequency interaction.
-
Mobile Responsiveness: Optimized text sizes and asset clarity to ensure the app is fully accessible and legible across all mobile devices.
4. Information Architecture & Content Strategy
-
Grouped Listings: Organized listed items into intuitive categories to reduce cognitive load, minimizing the time and frustration users might experience while browsing.
-
Activity Tracking: Created a dedicated page to track daily drink consumption, ensuring all item information remains accurate and dynamic.
-
Legal Copy: Transformed long-form terms and conditions into digestible, reader-friendly summaries.
AI detection problem and solutions
When SmartTaste AI first designed only to take pictures of food and drinks and tell users what is in the food and drink- after ideation and design work we start to think a solution for a problem where the AI may not understand everything, this is where we came in to a solution that users may need to train the AI and correct what the mirin is , this way user will have a great application that approximate be very close to 80 to 90 percent. That's where the edit button came to play where users can edit what the camera captures.
The difficulty in accurately detecting and analyzing drinks for an AI comes down to three main technical and biological problems: Opacity/Composition, Volume Estimation, and The "Hidden Calorie" Problem.
The Opacity and Composition Challenge
Unlike solid food, where boundaries are clear, liquids are homogenous mixtures that hide their components.
Mixed Ingredients: A single photo scan of a cocktail (like a margarita) or a custom coffee (like a latte) looks uniform. The AI must infer the presence and proportion of base ingredients (e.g., espresso, type of milk, alcohol) that are visually indistinct.
Layering and Viscosity: Different layers (like foam or crema) or varying viscosities (syrup sinking to the bottom) complicate image analysis, making it hard to use computer vision alone to separate ingredients.
The "Unknown Base": A glass of clear liquid could be water, clear soda, vodka, gin, or clear tonic. The AI requires OCR (reading labels) or user text/voice input to correctly disambiguate the base fluid, which is why your app uses a multimodal approach.
The Volume and Container Estimation Problem
Accurately calculating calories hinges on knowing the exact volume, which is extremely difficult with non-standard containers.
Non-Standard Vessels: Unlike standardized cups, drinks are served in various glasses, mugs, or oddly shaped bottles (e.g., a hand-blown glass, a large smoothie cup). The AI must perform complex Volume Estimation (as noted in your features) by recognizing the container shape and measuring the liquid fill line against the container's known capacity.
Perspective and Angle: A photo taken from a slight angle or close-up can dramatically skew the AI's calculation of the liquid depth and volume, leading to major calorie errors. The AI needs calibration/depth sensing, which is inherently hard in 2D images.
The "Hidden Calorie" (Macro Disaggregation) Problem
Primary difficulty
Added Sugar: Refined sugar (the "primary villain") or high-fructose corn syrup dissolves completely and is visually undetectable. The AI must prompt the user ("Did you add sugar / milk / flavoring?") because there is no way for computer vision to confirm 3 pumps of syrup vs. 1.
Alcohol Content: Alcohol is calorie-dense (7 calories/gram) and visually identical to water. For a complex cocktail, the AI must not only identify the drink type but also estimate the specific alcohol proof and pour size to calculate alcohol units and total calories, which requires external data lookup.
Satiety Failure: The lack of fiber or chewiness in liquids means the calories are often overlooked by the user and fail to trigger the feeling of fullness, reinforcing the need for the AI to provide visible, detailed macro context.
Solving the Opacity and Hidden Ingredient Challenge
The core solution is relying on the multimodal input and making the AI's best guess easily correctable by the user.
Interactive Refinement Toggles (On the AI Analysis & Correction screen). Instead of free text, use clean, toggle-based questions: e.g., "Milk Added?", "Syrup/Flavoring?", "Sweetener?".
Ingredient Disaggregation Model: AI uses image data to identify likely category (e.g., "coffee beverage"). It then uses external API data to present the most common mixed components (e.g., espresso, skim milk, vanilla syrup) and pre-populates the refinement fields.
For Photo Scan: If the AI confidence is below a threshold, trigger the Text/Voice Input modal immediately after the photo: "I see a clear liquid. Please tell me what it is."
Solving the Volume and Container Estimation Problem
The solution requires the AI to use context clues from the photo and utilize user guidance for better measurements.
Calibration Anchor: During the photo scan, prompt the user to briefly include a known object (like a coin or credit card) for a few seconds.
The AI can use this for scale to improve 3D depth and volume calculation.Computer Vision Volume Model: Train the model on hundreds of unique glass/cup types. The model identifies the vessel (e.g., "Pint Glass," "Mason Jar") and uses its known geometry and the liquid line to estimate volume more accurately than simple 2D analysis.
Guided Photo Tool: Implement an overlaid frame in the camera view instructing the user to center the drink and hold the phone at a slightly elevated, consistent angle (e.g., a 45-degree guide).
Solving the Hidden Calorie (Macro Disaggregation) Problem
The solution is to make the caloric information and healthier alternatives immediately obvious and actionable.
CRITICAL Macro Card Highlight: As defined in the blueprint, use a high-contrast visual treatment (e.g., red or bold font) and position the Added Sugar count immediately below total Carbs on the AI Analysis & Correction screen.
Conditional Macro Swap Logic: The recommendation engine runs an instant calculation: If (Calories > 400 OR AddedSugar > 30g), then Generate_Swap_Option(). This logic drives the Real-Time Macro Swap feature.
Gated Process: The Alcohol Confirmation Gate (custom modal) ensures compliance and accuracy by forcing a specific, separate flow for alcoholic drinks, allowing the AI to calculate Alcohol Units and factor in the additional 7 cal/gram into the total macro count.
More projects





