How to ask AI about your Mac screen by voice

Start with something you can check

Use a sample TextEdit document for your first walkthrough. Type a few lines and leave the window in front. The task is to create a PDF copy in a folder you can find again. You can verify both the suggested menu path and the finished file. Apple’s TextEdit guide

In HeyMate, connect a vision-capable AI provider. Microphone access lets you ask aloud, Screen Recording provides screen context, and Accessibility supports the hotkey and pointing. Connect your AI · Review Mac permissions

Ask for the next click

Hold Control + Option while you ask the question below, then release the keys when you finish speaking.
Check the label and position of the suggested control. Open the File menu yourself and look for Export as PDF.
If the walkthrough is waiting, say “next” or click the indicated target when you are ready. Ask about the save dialog if you need help choosing a location.
Open the PDF in Preview. Check that the text matches your sample and that the original document is still open in TextEdit.

Try this prompt

I want a PDF copy of this TextEdit document. Show me the first control to use. Wait while I click it, then help me check where the PDF will go.

If the assistant describes a control you cannot see, report what is visible. For example, “The File menu is open and I see Print. Where should I look for Export as PDF?” A guided turn can capture the updated screen.

Choose between talking and typing

Speaking is convenient when your hands are on the mouse. Typing can be easier in a shared office or when the question includes a precise error string. The HeyMate 1.0.90 update also supports typed screen explanations. For questions that are answerable from text alone, use text-only chat and paste the relevant part of the message. HeyMate 1.0.90 release notes

If a task needs a visible cue, say so. “Explain this setting” asks for an answer. “Point to the setting and wait” tells the assistant how to help you use the interface. That distinction matters when the same label appears in two places.

If the screen or microphone seems unavailable

If typed questions work but spoken ones fail, start by checking microphone permission and the app’s voice settings. If the assistant answers but cannot describe the visible app, bring the intended window forward, check Screen Recording permission, and confirm your model supports images. Screen guidance troubleshooting

Use the same method with a small app task you actually care about, such as finding an export option or making sense of a warning. If the screen contains someone else’s work or personal information, choose what to share first.

Try it with one small task.

HeyMate is free and open source for macOS 14.2 and later. Bring your own AI; provider limits and optional service fees apply. Builds are Developer ID signed and notarized by Apple.

Set up HeyMate on Mac