Agentic assistants: safe calendar and Gmail integrations

My claim: building safe, usable agentic assistants that act on calendars, Gmail and connected apps requires designing a permission-first runtime, separating planning from acting, and baking audit and revert flows into the execution layer from day one. This article gives a tight engineering checklist and runnable prompts to get there.

PERMISSION DESIGN FOR AGENTIC ASSISTANTS

Start with per-user OAuth and least-privilege scopes. Use user-bound OAuth rather than a shared service account except when an administrator explicitly requires domain delegation. Google’s documentation on granular scopes is the baseline; map every agent capability to a single OAuth scope and document the human-readable intent for that scope. Keep tokens out of prompts. Store them in a vault and inject credentials only at call-time inside an execution service that enforces policy.

We observed that teams who treat permissions as product UX, with clear consent screens and scope previews, get far fewer support tickets. One issue we encountered is token proliferation from naive refresh logic; centralize refresh and rotate refresh tokens regularly.

PLANNING VERSUS ACTING

Split the assistant into a planner and an actuator. The planner produces a discrete action plan: read recent threads, propose two meeting times, compose a draft email. The actuator executes only after a policy check and, for sensitive actions, a human confirmation step. This pattern keeps the model from improvising privileged operations and makes authorization auditable.

Practical trade-off: longer latency for human approval versus lower risk. For many calendar and email actions a 30 to 60 second human-in-the-loop pause is acceptable. For high-frequency tasks, batch approvals work better than per-action clicks.

TESTS, LOGS AND RECOVERY

Every action must produce an immutable audit entry containing the requested intent, the planner output, the policy decision, the actor call and the returned API response. Keep a replayable command format so you can replay and roll back. Provide a one-click revoke for the last N actions and a recovery flow that creates compensating events, for example cancelling an event and sending a corrective follow-up.

We observed that prompt-level guardrails fail without an external policy engine. In practice, models will suggest changes that look plausible but breach policy. An execution gate that refuses any write when confidence is below a calibrated threshold reduces these incidents.

CHECKLIST FOR PRODUCT AND ENGINEERING

1. Per-user OAuth with minimal scopes and explicit consent text. 2. Token vault and execution service that injects credentials at runtime. 3. Planner/actor separation with policy checks. 4. Human-in-the-loop escalation for sensitive actions. 5. Immutable audit logs and replayable action format. 6. Revert and compensating flows with clear UI affordances.

Link product UX to the workflow patterns described in Designing human-AI workflows and use the primer on agent differences in AI agents vs chatbots to justify the planner/actor split.

Try it yourself. The prompts below demonstrate planner output versus actuator-safe instructions. Expect terse JSON-like plans from the planner and short confirmations from the actuator.

This prompt asks the model to produce a constrained plan of candidate meeting times and a rationale; use it as the planner. Expect 2 to 3 candidate slots and one-sentence rationale.

You are a meeting planner. User has 3 free slots today: 10:00 AM, 2:30 PM, 4:00 PM. Return exactly three candidate meeting times in ISO format with one-line rationale for each, and a one-line privacy note saying whether the invite touches external emails.

This prompt is for the actuator. It expects a human-confirmation token and an explicit policy-check pass before creating the event.

Actuator: given user confirmation token X and planner plan Y, call Calendar.CreateEvent with fields {start,end,attendees,summary} only if policy_check(policy_id:calendar_write) returns pass. If policy_check fails, return the failure code and human action required.

A counterargument is that strict controls slow adoption. My estimate is that when teams enforce these patterns, early user retention improves because trust grows, even if initial activation is slower. The trade-off is clear: faster rollout without these controls creates measurable risk and higher remediation costs.