Rules Management

Overview

Rules Management allows you to define and manage moderation rules that automatically detect and handle inappropriate content. Configure rules once in the Dashboard, and they’re automatically applied to all messages sent through CometChat.

Using UI Kit or SDK? Once you configure rules in the Dashboard, they are automatically applied to all messages. No additional code is required - the UI Kit and SDK handle moderation seamlessly.

---

Quick Start

Enable moderation in under 2 minutes:

1. Open Rules Settings

   Login to CometChat Dashboard → Select your app → Moderation → Settings → Rules

2. Enable Default Rules

   Toggle on the rules you need (e.g., Profanity Filter, AI Image Moderation)

3. Test It

   Send a test message that violates the rule - it should be blocked automatically

---

Available Rules Summary

Text Rules

Rule	Description	Use Case
Profanity Filter	Blocks offensive language using keyword matching	General chat apps
Contact Details Filter	Blocks phone numbers	Marketplaces, dating apps
Email Filter	Blocks email addresses	Prevent off-platform contact
AI Toxicity Detection	AI-powered detection of toxic language	Community platforms
AI Spam Detection	Detects spam messages	High-traffic apps
AI Scam Detection	Identifies fraudulent messages	Marketplaces, finance apps
AI Platform Circumvention	Detects attempts to bypass rules	All apps

Image Rules

Rule	Description	Use Case
AI Image Moderation	Blocks explicit/unsafe images	All apps with image sharing
OpenAI Explicit Content	Detects nudity and sexual content	Social apps, dating
OpenAI Violence/Gore	Blocks graphic violence	Family-friendly apps
OpenAI Hate Symbols	Detects hate imagery	Community platforms
OpenAI Privacy Data	Blocks images with personal data	Finance, healthcare

Video Rules

Rule	Description	Use Case
AI Video Moderation	Blocks explicit/unsafe video content	Apps with video sharing

OpenAI Rules

Rule	Type	Description
Hate & Harassment	Text/Image	Detects hateful or harassing content
Explicit Content	Text/Image	Blocks sexual or graphic content
Privacy & Sensitive Info	Text/Image	Protects personal data
Spam & Scam	Text	Identifies fraudulent schemes
Violence & Terrorism	Text/Image	Blocks violent extremism
Self-Harm Content	Text/Image	Detects self-harm references
Impersonation & Fraud	Text	Prevents identity fraud
Minor Safety	Image	Protects minors from exploitation

---

Recommended Rules by Use Case

Social/Community App

Essential:

• Profanity Filter
• AI Toxicity Detection
• AI Image Moderation
• OpenAI Hate & Harassment

Recommended:

• AI Spam Detection
• OpenAI Explicit Content

Marketplace/Classifieds

Essential:

• Contact Details Filter
• Email Filter
• AI Scam Detection
• AI Platform Circumvention

Recommended:

• Profanity Filter
• AI Image Moderation

Dating App

Essential:

• Contact Details Filter
• Email Filter
• AI Image Moderation
• OpenAI Explicit Content

Recommended:

• AI Scam Detection
• OpenAI Privacy Data (Image)

Gaming/Entertainment

Essential:

• Profanity Filter
• AI Toxicity Detection
• AI Spam Detection

Recommended:

• OpenAI Hate & Harassment
• AI Image Moderation

Healthcare/Finance

Essential:

• OpenAI Privacy & Sensitive Info
• OpenAI Privacy Data (Image)
• AI Scam Detection

Recommended:

• Profanity Filter
• Contact Details Filter

---

Best Practices

Start with Defaults

Enable default rules first, then customize based on your needs. Don’t try to configure everything at once.

Test Before Launch

Test rules in a staging environment. Send test messages to verify rules work as expected.

Monitor Regularly

Check the Blocked Messages dashboard regularly to catch false positives.

Iterate & Refine

Review Rule Revisions to track changes and refine rules over time.

Tips for Effective Moderation

1. Layer your rules - Use multiple rules together (e.g., Profanity Filter + AI Toxicity) for better coverage
2. Adjust confidence levels - Lower confidence = more aggressive blocking, higher = fewer false positives
3. Use Flag action for borderline content - Instead of blocking, flag messages for manual review
4. Create custom keyword lists - Add industry-specific terms to the Lists Management

---

Default Rules

Default rules are pre-configured and ready to use. Simply toggle them on in the Dashboard.

Text Moderation

Profanity Filter

Automatically detects and blocks messages containing offensive language, profanity, or derogatory remarks using a predefined list of offensive keywords.

Example: Before enabling, profane messages are delivered (double ticks). After enabling, they’re blocked (single tick).

Contact Details Filter

Detects and blocks messages containing phone numbers to prevent sharing of private contact information.

Email Filter

Detects and blocks messages containing email addresses to prevent off-platform communication.

AI Message Toxicity

AI-powered detection of toxic, harmful, or inappropriate language including threats, harassment, and hate speech. Supports multiple languages.

AI Platform Circumvention

Detects attempts by users to bypass platform rules using sentence similarity analysis.

AI Scam Detection

Identifies scam-related messages including phishing attempts, fake offers, and fraudulent schemes.

AI Spam Detection

Detects and filters spam messages by analyzing content patterns and identifying unwanted communications.

Image Moderation

AI Image Moderation

Uses AI to identify and block images containing sensitive, explicit, or prohibited content.

OpenAI: Hate or Harassment

Detects hate symbols, extremist insignia, and harassing imagery.

OpenAI: Explicit or Sexual Content

Identifies nudity, explicit sexual content, or suggestive imagery.

OpenAI: Graphic Violence or Gore

Detects images of extreme violence, gore, or disturbing content.

OpenAI: Privacy or Personal Data

Identifies images containing personal data like IDs, addresses, or financial documents.

OpenAI: Self-Harm Content

Detects imagery suggesting self-harm or suicidal ideation.

OpenAI: Minor Safety

Detects exploitative imagery of minors or unsafe depictions of children.

OpenAI: Fraud or Scam Indicators

Flags manipulated or fraudulent images like fake IDs or doctored screenshots.

OpenAI: Terrorism or Extremism

Detects extremist propaganda, terrorist symbols, or violent ideologies.

Video Moderation

AI Video Moderation

Uses AI to identify and block videos containing sensitive, explicit, or prohibited content.

OpenAI Text Rules

OpenAI: Hate and Harassment (All Languages)

Detects hateful or harassing language toward individuals or groups using OpenAI moderation.

OpenAI: Privacy and Sensitive Info (All Languages)

Detects messages sharing personal or sensitive information without consent.

OpenAI: Explicit or Inappropriate Content (All Languages)

Identifies explicit sexual descriptions, graphic violence, or unsuitable text.

OpenAI: Spam and Scam (All Languages)

Detects spam messages, phishing attempts, and fraudulent schemes.

OpenAI: Violent or Terroristic Threats (All Languages)

Identifies content promoting violence or extremism.

OpenAI: Non-Consensual Sexual Content (All Languages)

Detects sexual exploitation, grooming, or non-consensual content.

OpenAI: Impersonation or Fraud (All Languages)

Identifies deceptive attempts to impersonate individuals or organizations.

OpenAI: Self-Harm or Suicidal Content (All Languages)

Detects messages suggesting self-harm or suicidal thoughts.

---

Rule Configuration

Filters

Filters narrow down which messages are checked by a rule based on sender or receiver attributes:

Filter Type	Properties
Sender	UID, Role, Name, Tags, Created date
Receiver	Name, GUID, Tags, Group type, Created date, Type (user/group)

Conditions

Conditions define what content triggers the rule:

Content Type	Available Conditions
Text/Custom	Keyword list, Word patterns, Toxicity, Sentiment, Sentence similarity
Image/Video	Violence, Gambling, Alcohol, Drugs, Nudity, Hate symbols, Unsafe content

You can set a confidence percentage for AI-based conditions to control sensitivity.

Actions

Actions determine what happens when content matches the conditions:

Action	Description
Block	Message is not delivered (default)
Flag	Message is delivered but flagged for review
Ban User	User is banned from the group
Kick User	User is removed from the group
Block User	User is blocked platform-wide

---

Managing Rules

Create Rule

1. Click Add in the Rules tab
2. Configure:
   • Name: Descriptive name for the rule
   • Rule ID: Unique identifier
   • Description: Purpose of the rule
   • Filter: Who the rule applies to
   • Condition: What triggers the rule
   • Action: What happens when triggered
3. Click Save
4. Enable the rule to start moderating

List Rules

All configured rules are displayed in the Rules tab with their name, status, and actions.

Update Rule

1. Click Edit in the action menu
2. Modify the rule settings
3. Click Save

Delete Rule

Click Delete in the action menu and confirm.

Rule Revisions

Track the history of changes made to a rule:

1. Click View in the action menu
2. Navigate to Rule History

---

Related Resources

Lists Management

Create custom keyword lists for your rules

Blocked Messages

View and manage blocked content

Flagged Messages

Review flagged content

SDK Integration

Handle moderation in custom UI