Privacy and Security Issues in Generative AI: GenAI Shield, a Prompt-Level Privacy and Security Analyzer

Abstract

Users of Generative AI (GenAI) systems routinely paste personal data, passwords, API keys and internal documents into prompts, while the systems themselves face attacks such as prompt injection, and users have few tools that check a prompt before it is submitted. This paper presents the design and evaluation framework of GenAI Shield, a prompt-level privacy and security analyzer that combines pattern-based detection of personal data and credentials, rule-based detection of confidential markers, prompt-injection and suspicious content, a transparent project-specific risk score (0–100, four levels), and a rule-driven recommendation engine. The system (Python/Flask, SQLite, Chart.js) is under development; no module has yet been implemented, deployed or tested, and this paper reports the proposed design only. A synthetic Stage 1 dataset of 200 prompts in six categories was used to demonstrate the evaluation method (synthetic data was used for Stage 1 because the prototype is incomplete, and will be replaced by measured results in Stage 2); the detector outcomes in it are assumed and illustrative. Under that illustration, precision is 0.95, recall 0.79 and F1 0.86 against synthetic labels, with structured categories (personal data, credentials) flagged at 94.55% and semantic categories (confidential, injection, suspicious content) at 68.24%. The contribution is an architectural and methodological integration of privacy detection, threat detection, scoring and advice for GenAI prompts, reported with an explicit account of what is proposed versus what is measured.