AI-Built App Test Matrix for Non-Developers

01

What this matrix can and cannot prove

A test matrix turns vague confidence into a written record of what you checked. It does not certify that an app is secure, legally compliant, production-ready, or suitable for every user.

• A passing test proves only that the stated scenario behaved as expected at that time.
• A build or type check can pass while a real user flow, database policy, payment event, or mobile layout is broken.
• Record tests that were not run and assumptions that still need manual review.
• Use a separate security review for secrets, authentication, RLS, payments, and provider settings.

02

Happy-path testing

Start with the normal journey a real user should complete, from the first screen to the intended result.

• Open the app as a new visitor and confirm the main promise is understandable.
• Complete the primary action from start to finish.
• Confirm success is visible and understandable to the user.
• Refresh, return later, and confirm the saved state is what you expect.
• Repeat the flow after deployment, not only on a local machine.

03

Empty, invalid, and duplicate input

Most demos are tested with perfect input. Real users leave fields blank, paste unexpected values, click twice, and lose connection.

• Submit required fields empty and check for a helpful message.
• Try short, long, malformed, and unexpected input.
• Click the submit or payment button twice and check for duplicate records or charges.
• Refresh during a request and check whether the app handles the interruption clearly.
• Confirm error messages do not expose secrets, stack traces, or private data.

04

Auth and second-user testing

A feature that works for the first account is not enough. Use separate accounts to check ownership and access boundaries.

• Create two test users with different accounts.
• Confirm user A cannot read, edit, or delete user B’s private records.
• Test sign-up, sign-in, sign-out, password reset, and expired-session behavior.
• Try visiting a private URL while signed out.
• Check what happens when a user is deleted, disabled, or has no profile row.

05

Supabase, RLS, and data isolation

The interface hiding a record is not the same as the database refusing access. Verify the database rules and the application behavior separately.

• List each table and decide whether its data is public, user-owned, or admin-only.
• Confirm Row Level Security is enabled where private data requires it.
• Test reads, inserts, updates, and deletes as more than one user.
• Check storage buckets as well as database tables.
• Verify service-role credentials are used only on trusted server-side paths.

06

Payment and webhook states

Payment screens are only one part of the system. Test the provider event and the entitlement change that follows it.

• Test a successful payment in provider test mode.
• Test a failed, cancelled, refunded, and delayed payment state.
• Send the same webhook event more than once and check for duplicate access or records.
• Confirm signatures are verified before an event changes access.
• Check that cancellations and refunds remove or change entitlements as intended.
• Do not switch to live mode until provider-side and application-side tests are documented.

07

Mobile and slow-network behavior

A desktop browser on a fast connection is only one environment. Test the smallest screen and the least convenient conditions you expect to support.

• Test the main flow on a real phone or a narrow viewport.
• Check keyboard behavior, tap targets, scrolling, loading states, and error visibility.
• Throttle the connection or test with delayed responses.
• Confirm a slow request cannot be submitted repeatedly by accident.
• Check that important content is still readable without relying on hover.

08

Deployment and environment variables

A successful deployment means the hosting platform completed its process. It does not prove that production configuration or every live user flow is correct.

• Record the commit, build command, publish directory, and environment used.
• Check that required production variables exist without exposing their values.
• Test the production domain, redirects, forms, functions, and API calls.
• Review deployment logs for warnings that could affect the live app.
• Keep a known-good deployment or commit so rollback is understandable.

09

Recovery and rollback

Testing should leave you with evidence and a recovery point, not just a green status message.

• Confirm the current GitHub branch and working tree status.
• Record the last known-good commit before a large change.
• Save screenshots, error messages, and test results when something fails.
• Ask AI to explain changed files before asking it to repair several problems at once.
• Verify the fix after recovery before pushing or deploying again.

10

Evidence log and Codex prompt

For each test, record: scenario, expected result, actual result, evidence, date, environment, and unknowns. Then ask Codex to organize the matrix without inventing results.