diff --git a/.gitignore b/.gitignore index 6f781509d5f4c292645a2425e9d6291a5aa33b41..b71d058d0661cfe06e439d19595f4079f6e487dc 100644 --- a/.gitignore +++ b/.gitignore @@ -7,3 +7,7 @@ venv/ # Ignoring data directory to prevent pushing binary index files data/index/ data/sessions/ + +# Ignore local backup directories +backup*/ +*backup*/ diff --git a/backup_2026-05-04/.env.example b/backup_2026-05-04/.env.example deleted file mode 100644 index de3084a512d5ffca421b1cb7e4fad59fe084ab02..0000000000000000000000000000000000000000 --- a/backup_2026-05-04/.env.example +++ /dev/null @@ -1,15 +0,0 @@ -LLM_PROVIDER=groq -GROQ_API_KEY=your_groq_api_key -GROQ_MODEL=llama-3.1-8b-instant -HF_API_KEY=your_huggingface_api_key -HF_MODEL=meta-llama/Llama-3.1-8B-Instruct -EMBEDDING_MODEL=BAAI/bge-small-en-v1.5 -DOCS_DIR=docs -INDEX_DIR=data/index -SESSIONS_DIR=data/sessions -TOP_K=4 -CORS_ALLOW_ORIGINS=* -API_KEY= -RATE_LIMIT_REQUESTS=60 -RATE_LIMIT_WINDOW_SECONDS=60 -RAG_API_URL= diff --git a/backup_2026-05-04/.gitattributes b/backup_2026-05-04/.gitattributes deleted file mode 100644 index f18a35d12ad3bd5aff5f573ece54f1bd056249aa..0000000000000000000000000000000000000000 --- a/backup_2026-05-04/.gitattributes +++ /dev/null @@ -1,2 +0,0 @@ -*.lottie filter=lfs diff=lfs merge=lfs -text -*.pdf filter=lfs diff=lfs merge=lfs -text diff --git a/backup_2026-05-04/.gitignore b/backup_2026-05-04/.gitignore deleted file mode 100644 index 6f781509d5f4c292645a2425e9d6291a5aa33b41..0000000000000000000000000000000000000000 --- a/backup_2026-05-04/.gitignore +++ /dev/null @@ -1,9 +0,0 @@ -.env -__pycache__/ -.pytest_cache/ -.mypy_cache/ -.venv/ -venv/ -# Ignoring data directory to prevent pushing binary index files -data/index/ -data/sessions/ diff --git a/backup_2026-05-04/CONFIG_NOTES.md b/backup_2026-05-04/CONFIG_NOTES.md deleted file mode 100644 index 5517ac20537f1769b1bbc6d630b79b85e67b712f..0000000000000000000000000000000000000000 --- a/backup_2026-05-04/CONFIG_NOTES.md +++ /dev/null @@ -1,38 +0,0 @@ -# Configuration Backup - Martechsol RAG Chatbot -**Date:** 2026-04-28 -**Source:** C:\Users\DELL\Desktop\RAG_backup2 -**Destination:** D:\RAG_Backup_2026_04_28 - -## Core Settings (from app/core/config.py) -- **App Name:** Fast RAG Chatbot -- **LLM Provider:** groq (default) -- **Groq Model:** llama-3.1-8b-instant -- **HF Model:** meta-llama/Llama-3.1-8B-Instruct -- **Embedding Model:** BAAI/bge-small-en-v1.5 -- **Docs Directory:** docs/ -- **Index Directory:** data/index/ -- **Sessions Directory:** data/sessions/ -- **Chunk Size:** 420 tokens -- **Overlap:** 80 tokens -- **Top K:** 4 - -## Admin Credentials -- **Username:** martech_admin -- **Password:** martech_admin_303 -- **OTP Expiry:** 300 seconds (5 minutes) - -## Security -- **Auth Method:** HTTP Basic Auth + OTP (email-based) -- **Email for OTP:** randomjoedown@gmail.com -- **SMTP Server:** smtp.gmail.com (Port 465) - -## API / Integration -- **Endpoint:** /api/chat -- **Widget Endpoint:** /widget -- **Admin Endpoint:** /admin -- **CORS:** Allowed for all (*) for testing - -## Infrastructure -- **Hugging Face Path:** /data (for persistent storage) -- **Local Path:** data/ (for local storage) -- **Dockerfile:** Based on python:3.10-slim, serves on port 7860 diff --git a/backup_2026-05-04/CUSTOMIZATION_GUIDE.md b/backup_2026-05-04/CUSTOMIZATION_GUIDE.md deleted file mode 100644 index 22d70e37fd9c17d9a66426cec7158d449be17064..0000000000000000000000000000000000000000 --- a/backup_2026-05-04/CUSTOMIZATION_GUIDE.md +++ /dev/null @@ -1,55 +0,0 @@ -# Martechsol Assistant Customization Guide - -This guide explains how to customize the visual appearance and behavior of your Gradio-based chat interface. - -## 1. Modifying the UI Layout (app/ui_gradio.py) - -The interface is built using `gradio.Blocks`. You can modify the structure in the `with gr.Blocks(...) as demo:` section. - -### Common Components: -- **`chatbot`**: The main chat window. We use `elem_id="chatbot-window"` for CSS targeting. -- **`msg`**: The text input box. -- **`send`**: The primary action button. - -## 2. Custom CSS (app/ui_gradio.py) - -At the top of `app/ui_gradio.py`, there is a `custom_css` string. This is the most powerful way to change the look of your app. - -### How to target elements: -We use `elem_id` in Python to give elements stable names. - -Example: -```python -chatbot = gr.Chatbot(elem_id="my-custom-chat") -``` -Then in CSS: -```css -#my-custom-chat { - background-color: #f0f0f0; -} -``` - -### Important Selectors: -- **`footer`**: Targets the Gradio branding at the bottom. -- **`.gradio-container`**: The main outer wrapper of your app. -- **`[data-testid="user"]`**: Targets user messages. -- **`[data-testid="bot"]`**: Targets assistant messages. - -## 3. Persistent Data on Hugging Face - -Your data is stored in the `/data` volume. -- **Sessions**: `/data/sessions/` -- **FAISS Index**: `/data/index/` - -To manage these, you can use the **Admin Panel** at `/admin`. - -## 4. Tips for Widget Embedding - -If you are embedding this as an iframe (like in `addon.html`): -1. **Remove redundant headers**: We hide the Markdown titles in the CSS (`#title-area { display: none; }`) so the iframe looks like a part of your page. -2. **Handle height**: We set `height: auto !important` and `min-height: 40px !important` on the chatbot so it doesn't take up 500px by default. -3. **Transparent Background**: You can add `body { background-color: transparent !important; }` to the `custom_css` if your theme supports it. - -## 5. Deployment - -Whenever you save a file, the changes are automatically pushed to your Hugging Face Space. The Space will rebuild automatically (takes about 1-2 minutes). diff --git a/backup_2026-05-04/Dockerfile b/backup_2026-05-04/Dockerfile deleted file mode 100644 index b2137aa0e63b826e988226e4902e1d03496b7078..0000000000000000000000000000000000000000 --- a/backup_2026-05-04/Dockerfile +++ /dev/null @@ -1,31 +0,0 @@ -FROM python:3.11-slim - -# Set environment variables -ENV PYTHONDONTWRITEBYTECODE=1 -ENV PYTHONUNBUFFERED=1 -ENV HOME=/home/user - -# Create a non-root user for Hugging Face -RUN useradd -m -u 1000 user -WORKDIR $HOME/app - -# Pre-create data directories with correct ownership -RUN mkdir -p $HOME/app/data/index $HOME/app/data/sessions && \ - chown -R user:user $HOME/app - -# Switch to non-root user -USER user -ENV PATH=$HOME/.local/bin:$PATH - -# Install dependencies -COPY --chown=user requirements.txt . -RUN pip install --no-cache-dir --user -r requirements.txt - -# Copy application code -COPY --chown=user . . - -# Expose port -EXPOSE 7860 - -# Start application -CMD ["python", "-m", "uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "7860", "--workers", "1"] diff --git a/backup_2026-05-04/PHPMailer/Exception.php b/backup_2026-05-04/PHPMailer/Exception.php deleted file mode 100644 index 09c1a2cfefaa986c846b41743e856c945beff3cd..0000000000000000000000000000000000000000 --- a/backup_2026-05-04/PHPMailer/Exception.php +++ /dev/null @@ -1,40 +0,0 @@ - - * @author Jim Jagielski (jimjag) - * @author Andy Prevost (codeworxtech) - * @author Brent R. Matzelle (original founder) - * @copyright 2012 - 2020 Marcus Bointon - * @copyright 2010 - 2012 Jim Jagielski - * @copyright 2004 - 2009 Andy Prevost - * @license https://www.gnu.org/licenses/old-licenses/lgpl-2.1.html GNU Lesser General Public License - * @note This program is distributed in the hope that it will be useful - WITHOUT - * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or - * FITNESS FOR A PARTICULAR PURPOSE. - */ - -namespace PHPMailer\PHPMailer; - -/** - * PHPMailer exception handler. - * - * @author Marcus Bointon - */ -class Exception extends \Exception -{ - /** - * Prettify error message output. - * - * @return string - */ - public function errorMessage() - { - return '' . htmlspecialchars($this->getMessage(), ENT_COMPAT | ENT_HTML401) . "
\n"; - } -} diff --git a/backup_2026-05-04/PHPMailer/PHPMailer.php b/backup_2026-05-04/PHPMailer/PHPMailer.php deleted file mode 100644 index 31731594f12ec7bb6bd2eca0a990c91233f2137a..0000000000000000000000000000000000000000 --- a/backup_2026-05-04/PHPMailer/PHPMailer.php +++ /dev/null @@ -1,5250 +0,0 @@ - - * @author Jim Jagielski (jimjag) - * @author Andy Prevost (codeworxtech) - * @author Brent R. Matzelle (original founder) - * @copyright 2012 - 2020 Marcus Bointon - * @copyright 2010 - 2012 Jim Jagielski - * @copyright 2004 - 2009 Andy Prevost - * @license https://www.gnu.org/licenses/old-licenses/lgpl-2.1.html GNU Lesser General Public License - * @note This program is distributed in the hope that it will be useful - WITHOUT - * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or - * FITNESS FOR A PARTICULAR PURPOSE. - */ - -namespace PHPMailer\PHPMailer; - -/** - * PHPMailer - PHP email creation and transport class. - * - * @author Marcus Bointon (Synchro/coolbru) - * @author Jim Jagielski (jimjag) - * @author Andy Prevost (codeworxtech) - * @author Brent R. Matzelle (original founder) - */ -class PHPMailer -{ - const CHARSET_ASCII = 'us-ascii'; - const CHARSET_ISO88591 = 'iso-8859-1'; - const CHARSET_UTF8 = 'utf-8'; - - const CONTENT_TYPE_PLAINTEXT = 'text/plain'; - const CONTENT_TYPE_TEXT_CALENDAR = 'text/calendar'; - const CONTENT_TYPE_TEXT_HTML = 'text/html'; - const CONTENT_TYPE_MULTIPART_ALTERNATIVE = 'multipart/alternative'; - const CONTENT_TYPE_MULTIPART_MIXED = 'multipart/mixed'; - const CONTENT_TYPE_MULTIPART_RELATED = 'multipart/related'; - - const ENCODING_7BIT = '7bit'; - const ENCODING_8BIT = '8bit'; - const ENCODING_BASE64 = 'base64'; - const ENCODING_BINARY = 'binary'; - const ENCODING_QUOTED_PRINTABLE = 'quoted-printable'; - - const ENCRYPTION_STARTTLS = 'tls'; - const ENCRYPTION_SMTPS = 'ssl'; - - const ICAL_METHOD_REQUEST = 'REQUEST'; - const ICAL_METHOD_PUBLISH = 'PUBLISH'; - const ICAL_METHOD_REPLY = 'REPLY'; - const ICAL_METHOD_ADD = 'ADD'; - const ICAL_METHOD_CANCEL = 'CANCEL'; - const ICAL_METHOD_REFRESH = 'REFRESH'; - const ICAL_METHOD_COUNTER = 'COUNTER'; - const ICAL_METHOD_DECLINECOUNTER = 'DECLINECOUNTER'; - - /** - * Email priority. - * Options: null (default), 1 = High, 3 = Normal, 5 = low. - * When null, the header is not set at all. - * - * @var int|null - */ - public $Priority; - - /** - * The character set of the message. - * - * @var string - */ - public $CharSet = self::CHARSET_ISO88591; - - /** - * The MIME Content-type of the message. - * - * @var string - */ - public $ContentType = self::CONTENT_TYPE_PLAINTEXT; - - /** - * The message encoding. - * Options: "8bit", "7bit", "binary", "base64", and "quoted-printable". - * - * @var string - */ - public $Encoding = self::ENCODING_8BIT; - - /** - * Holds the most recent mailer error message. - * - * @var string - */ - public $ErrorInfo = ''; - - /** - * The From email address for the message. - * - * @var string - */ - public $From = ''; - - /** - * The From name of the message. - * - * @var string - */ - public $FromName = ''; - - /** - * The envelope sender of the message. - * This will usually be turned into a Return-Path header by the receiver, - * and is the address that bounces will be sent to. - * If not empty, will be passed via `-f` to sendmail or as the 'MAIL FROM' value over SMTP. - * - * @var string - */ - public $Sender = ''; - - /** - * The Subject of the message. - * - * @var string - */ - public $Subject = ''; - - /** - * An HTML or plain text message body. - * If HTML then call isHTML(true). - * - * @var string - */ - public $Body = ''; - - /** - * The plain-text message body. - * This body can be read by mail clients that do not have HTML email - * capability such as mutt & Eudora. - * Clients that can read HTML will view the normal Body. - * - * @var string - */ - public $AltBody = ''; - - /** - * An iCal message part body. - * Only supported in simple alt or alt_inline message types - * To generate iCal event structures, use classes like EasyPeasyICS or iCalcreator. - * - * @see https://kigkonsult.se/iCalcreator/ - * - * @var string - */ - public $Ical = ''; - - /** - * Value-array of "method" in Contenttype header "text/calendar" - * - * @var string[] - */ - protected static $IcalMethods = [ - self::ICAL_METHOD_REQUEST, - self::ICAL_METHOD_PUBLISH, - self::ICAL_METHOD_REPLY, - self::ICAL_METHOD_ADD, - self::ICAL_METHOD_CANCEL, - self::ICAL_METHOD_REFRESH, - self::ICAL_METHOD_COUNTER, - self::ICAL_METHOD_DECLINECOUNTER, - ]; - - /** - * The complete compiled MIME message body. - * - * @var string - */ - protected $MIMEBody = ''; - - /** - * The complete compiled MIME message headers. - * - * @var string - */ - protected $MIMEHeader = ''; - - /** - * Extra headers that createHeader() doesn't fold in. - * - * @var string - */ - protected $mailHeader = ''; - - /** - * Word-wrap the message body to this number of chars. - * Set to 0 to not wrap. A useful value here is 78, for RFC2822 section 2.1.1 compliance. - * - * @see static::STD_LINE_LENGTH - * - * @var int - */ - public $WordWrap = 0; - - /** - * Which method to use to send mail. - * Options: "mail", "sendmail", or "smtp". - * - * @var string - */ - public $Mailer = 'mail'; - - /** - * The path to the sendmail program. - * - * @var string - */ - public $Sendmail = '/usr/sbin/sendmail'; - - /** - * Whether mail() uses a fully sendmail-compatible MTA. - * One which supports sendmail's "-oi -f" options. - * - * @var bool - */ - public $UseSendmailOptions = true; - - /** - * The email address that a reading confirmation should be sent to, also known as read receipt. - * - * @var string - */ - public $ConfirmReadingTo = ''; - - /** - * The hostname to use in the Message-ID header and as default HELO string. - * If empty, PHPMailer attempts to find one with, in order, - * $_SERVER['SERVER_NAME'], gethostname(), php_uname('n'), or the value - * 'localhost.localdomain'. - * - * @see PHPMailer::$Helo - * - * @var string - */ - public $Hostname = ''; - - /** - * An ID to be used in the Message-ID header. - * If empty, a unique id will be generated. - * You can set your own, but it must be in the format "", - * as defined in RFC5322 section 3.6.4 or it will be ignored. - * - * @see https://www.rfc-editor.org/rfc/rfc5322#section-3.6.4 - * - * @var string - */ - public $MessageID = ''; - - /** - * The message Date to be used in the Date header. - * If empty, the current date will be added. - * - * @var string - */ - public $MessageDate = ''; - - /** - * SMTP hosts. - * Either a single hostname or multiple semicolon-delimited hostnames. - * You can also specify a different port - * for each host by using this format: [hostname:port] - * (e.g. "smtp1.example.com:25;smtp2.example.com"). - * You can also specify encryption type, for example: - * (e.g. "tls://smtp1.example.com:587;ssl://smtp2.example.com:465"). - * Hosts will be tried in order. - * - * @var string - */ - public $Host = 'localhost'; - - /** - * The default SMTP server port. - * - * @var int - */ - public $Port = 25; - - /** - * The SMTP HELO/EHLO name used for the SMTP connection. - * Default is $Hostname. If $Hostname is empty, PHPMailer attempts to find - * one with the same method described above for $Hostname. - * - * @see PHPMailer::$Hostname - * - * @var string - */ - public $Helo = ''; - - /** - * What kind of encryption to use on the SMTP connection. - * Options: '', static::ENCRYPTION_STARTTLS, or static::ENCRYPTION_SMTPS. - * - * @var string - */ - public $SMTPSecure = ''; - - /** - * Whether to enable TLS encryption automatically if a server supports it, - * even if `SMTPSecure` is not set to 'tls'. - * Be aware that in PHP >= 5.6 this requires that the server's certificates are valid. - * - * @var bool - */ - public $SMTPAutoTLS = true; - - /** - * Whether to use SMTP authentication. - * Uses the Username and Password properties. - * - * @see PHPMailer::$Username - * @see PHPMailer::$Password - * - * @var bool - */ - public $SMTPAuth = false; - - /** - * Options array passed to stream_context_create when connecting via SMTP. - * - * @var array - */ - public $SMTPOptions = []; - - /** - * SMTP username. - * - * @var string - */ - public $Username = ''; - - /** - * SMTP password. - * - * @var string - */ - public $Password = ''; - - /** - * SMTP authentication type. Options are CRAM-MD5, LOGIN, PLAIN, XOAUTH2. - * If not specified, the first one from that list that the server supports will be selected. - * - * @var string - */ - public $AuthType = ''; - - /** - * SMTP SMTPXClient command attributes - * - * @var array - */ - protected $SMTPXClient = []; - - /** - * An implementation of the PHPMailer OAuthTokenProvider interface. - * - * @var OAuthTokenProvider - */ - protected $oauth; - - /** - * The SMTP server timeout in seconds. - * Default of 5 minutes (300sec) is from RFC2821 section 4.5.3.2. - * - * @var int - */ - public $Timeout = 300; - - /** - * Comma separated list of DSN notifications - * 'NEVER' under no circumstances a DSN must be returned to the sender. - * If you use NEVER all other notifications will be ignored. - * 'SUCCESS' will notify you when your mail has arrived at its destination. - * 'FAILURE' will arrive if an error occurred during delivery. - * 'DELAY' will notify you if there is an unusual delay in delivery, but the actual - * delivery's outcome (success or failure) is not yet decided. - * - * @see https://www.rfc-editor.org/rfc/rfc3461.html#section-4.1 for more information about NOTIFY - */ - public $dsn = ''; - - /** - * SMTP class debug output mode. - * Debug output level. - * Options: - * @see SMTP::DEBUG_OFF: No output - * @see SMTP::DEBUG_CLIENT: Client messages - * @see SMTP::DEBUG_SERVER: Client and server messages - * @see SMTP::DEBUG_CONNECTION: As SERVER plus connection status - * @see SMTP::DEBUG_LOWLEVEL: Noisy, low-level data output, rarely needed - * - * @see SMTP::$do_debug - * - * @var int - */ - public $SMTPDebug = 0; - - /** - * How to handle debug output. - * Options: - * * `echo` Output plain-text as-is, appropriate for CLI - * * `html` Output escaped, line breaks converted to `
`, appropriate for browser output - * * `error_log` Output to error log as configured in php.ini - * By default PHPMailer will use `echo` if run from a `cli` or `cli-server` SAPI, `html` otherwise. - * Alternatively, you can provide a callable expecting two params: a message string and the debug level: - * - * ```php - * $mail->Debugoutput = function($str, $level) {echo "debug level $level; message: $str";}; - * ``` - * - * Alternatively, you can pass in an instance of a PSR-3 compatible logger, though only `debug` - * level output is used: - * - * ```php - * $mail->Debugoutput = new myPsr3Logger; - * ``` - * - * @see SMTP::$Debugoutput - * - * @var string|callable|\Psr\Log\LoggerInterface - */ - public $Debugoutput = 'echo'; - - /** - * Whether to keep the SMTP connection open after each message. - * If this is set to true then the connection will remain open after a send, - * and closing the connection will require an explicit call to smtpClose(). - * It's a good idea to use this if you are sending multiple messages as it reduces overhead. - * See the mailing list example for how to use it. - * - * @var bool - */ - public $SMTPKeepAlive = false; - - /** - * Whether to split multiple to addresses into multiple messages - * or send them all in one message. - * Only supported in `mail` and `sendmail` transports, not in SMTP. - * - * @var bool - * - * @deprecated 6.0.0 PHPMailer isn't a mailing list manager! - */ - public $SingleTo = false; - - /** - * Storage for addresses when SingleTo is enabled. - * - * @var array - */ - protected $SingleToArray = []; - - /** - * Whether to generate VERP addresses on send. - * Only applicable when sending via SMTP. - * - * @see https://en.wikipedia.org/wiki/Variable_envelope_return_path - * @see https://www.postfix.org/VERP_README.html Postfix VERP info - * - * @var bool - */ - public $do_verp = false; - - /** - * Whether to allow sending messages with an empty body. - * - * @var bool - */ - public $AllowEmpty = false; - - /** - * DKIM selector. - * - * @var string - */ - public $DKIM_selector = ''; - - /** - * DKIM Identity. - * Usually the email address used as the source of the email. - * - * @var string - */ - public $DKIM_identity = ''; - - /** - * DKIM passphrase. - * Used if your key is encrypted. - * - * @var string - */ - public $DKIM_passphrase = ''; - - /** - * DKIM signing domain name. - * - * @example 'example.com' - * - * @var string - */ - public $DKIM_domain = ''; - - /** - * DKIM Copy header field values for diagnostic use. - * - * @var bool - */ - public $DKIM_copyHeaderFields = true; - - /** - * DKIM Extra signing headers. - * - * @example ['List-Unsubscribe', 'List-Help'] - * - * @var array - */ - public $DKIM_extraHeaders = []; - - /** - * DKIM private key file path. - * - * @var string - */ - public $DKIM_private = ''; - - /** - * DKIM private key string. - * - * If set, takes precedence over `$DKIM_private`. - * - * @var string - */ - public $DKIM_private_string = ''; - - /** - * Callback Action function name. - * - * The function that handles the result of the send email action. - * It is called out by send() for each email sent. - * - * Value can be any php callable: https://www.php.net/is_callable - * - * Parameters: - * bool $result result of the send action - * array $to email addresses of the recipients - * array $cc cc email addresses - * array $bcc bcc email addresses - * string $subject the subject - * string $body the email body - * string $from email address of sender - * string $extra extra information of possible use - * "smtp_transaction_id' => last smtp transaction id - * - * @var string - */ - public $action_function = ''; - - /** - * What to put in the X-Mailer header. - * Options: An empty string for PHPMailer default, whitespace/null for none, or a string to use. - * - * @var string|null - */ - public $XMailer = ''; - - /** - * Which validator to use by default when validating email addresses. - * May be a callable to inject your own validator, but there are several built-in validators. - * The default validator uses PHP's FILTER_VALIDATE_EMAIL filter_var option. - * - * @see PHPMailer::validateAddress() - * - * @var string|callable - */ - public static $validator = 'php'; - - /** - * An instance of the SMTP sender class. - * - * @var SMTP - */ - protected $smtp; - - /** - * The array of 'to' names and addresses. - * - * @var array - */ - protected $to = []; - - /** - * The array of 'cc' names and addresses. - * - * @var array - */ - protected $cc = []; - - /** - * The array of 'bcc' names and addresses. - * - * @var array - */ - protected $bcc = []; - - /** - * The array of reply-to names and addresses. - * - * @var array - */ - protected $ReplyTo = []; - - /** - * An array of all kinds of addresses. - * Includes all of $to, $cc, $bcc. - * - * @see PHPMailer::$to - * @see PHPMailer::$cc - * @see PHPMailer::$bcc - * - * @var array - */ - protected $all_recipients = []; - - /** - * An array of names and addresses queued for validation. - * In send(), valid and non duplicate entries are moved to $all_recipients - * and one of $to, $cc, or $bcc. - * This array is used only for addresses with IDN. - * - * @see PHPMailer::$to - * @see PHPMailer::$cc - * @see PHPMailer::$bcc - * @see PHPMailer::$all_recipients - * - * @var array - */ - protected $RecipientsQueue = []; - - /** - * An array of reply-to names and addresses queued for validation. - * In send(), valid and non duplicate entries are moved to $ReplyTo. - * This array is used only for addresses with IDN. - * - * @see PHPMailer::$ReplyTo - * - * @var array - */ - protected $ReplyToQueue = []; - - /** - * The array of attachments. - * - * @var array - */ - protected $attachment = []; - - /** - * The array of custom headers. - * - * @var array - */ - protected $CustomHeader = []; - - /** - * The most recent Message-ID (including angular brackets). - * - * @var string - */ - protected $lastMessageID = ''; - - /** - * The message's MIME type. - * - * @var string - */ - protected $message_type = ''; - - /** - * The array of MIME boundary strings. - * - * @var array - */ - protected $boundary = []; - - /** - * The array of available text strings for the current language. - * - * @var array - */ - protected $language = []; - - /** - * The number of errors encountered. - * - * @var int - */ - protected $error_count = 0; - - /** - * The S/MIME certificate file path. - * - * @var string - */ - protected $sign_cert_file = ''; - - /** - * The S/MIME key file path. - * - * @var string - */ - protected $sign_key_file = ''; - - /** - * The optional S/MIME extra certificates ("CA Chain") file path. - * - * @var string - */ - protected $sign_extracerts_file = ''; - - /** - * The S/MIME password for the key. - * Used only if the key is encrypted. - * - * @var string - */ - protected $sign_key_pass = ''; - - /** - * Whether to throw exceptions for errors. - * - * @var bool - */ - protected $exceptions = false; - - /** - * Unique ID used for message ID and boundaries. - * - * @var string - */ - protected $uniqueid = ''; - - /** - * The PHPMailer Version number. - * - * @var string - */ - const VERSION = '6.9.3'; - - /** - * Error severity: message only, continue processing. - * - * @var int - */ - const STOP_MESSAGE = 0; - - /** - * Error severity: message, likely ok to continue processing. - * - * @var int - */ - const STOP_CONTINUE = 1; - - /** - * Error severity: message, plus full stop, critical error reached. - * - * @var int - */ - const STOP_CRITICAL = 2; - - /** - * The SMTP standard CRLF line break. - * If you want to change line break format, change static::$LE, not this. - */ - const CRLF = "\r\n"; - - /** - * "Folding White Space" a white space string used for line folding. - */ - const FWS = ' '; - - /** - * SMTP RFC standard line ending; Carriage Return, Line Feed. - * - * @var string - */ - protected static $LE = self::CRLF; - - /** - * The maximum line length supported by mail(). - * - * Background: mail() will sometimes corrupt messages - * with headers longer than 65 chars, see #818. - * - * @var int - */ - const MAIL_MAX_LINE_LENGTH = 63; - - /** - * The maximum line length allowed by RFC 2822 section 2.1.1. - * - * @var int - */ - const MAX_LINE_LENGTH = 998; - - /** - * The lower maximum line length allowed by RFC 2822 section 2.1.1. - * This length does NOT include the line break - * 76 means that lines will be 77 or 78 chars depending on whether - * the line break format is LF or CRLF; both are valid. - * - * @var int - */ - const STD_LINE_LENGTH = 76; - - /** - * Constructor. - * - * @param bool $exceptions Should we throw external exceptions? - */ - public function __construct($exceptions = null) - { - if (null !== $exceptions) { - $this->exceptions = (bool) $exceptions; - } - //Pick an appropriate debug output format automatically - $this->Debugoutput = (strpos(PHP_SAPI, 'cli') !== false ? 'echo' : 'html'); - } - - /** - * Destructor. - */ - public function __destruct() - { - //Close any open SMTP connection nicely - $this->smtpClose(); - } - - /** - * Call mail() in a safe_mode-aware fashion. - * Also, unless sendmail_path points to sendmail (or something that - * claims to be sendmail), don't pass params (not a perfect fix, - * but it will do). - * - * @param string $to To - * @param string $subject Subject - * @param string $body Message Body - * @param string $header Additional Header(s) - * @param string|null $params Params - * - * @return bool - */ - private function mailPassthru($to, $subject, $body, $header, $params) - { - //Check overloading of mail function to avoid double-encoding - if ((int)ini_get('mbstring.func_overload') & 1) { // phpcs:ignore PHPCompatibility.IniDirectives.RemovedIniDirectives.mbstring_func_overloadDeprecated - $subject = $this->secureHeader($subject); - } else { - $subject = $this->encodeHeader($this->secureHeader($subject)); - } - //Calling mail() with null params breaks - $this->edebug('Sending with mail()'); - $this->edebug('Sendmail path: ' . ini_get('sendmail_path')); - $this->edebug("Envelope sender: {$this->Sender}"); - $this->edebug("To: {$to}"); - $this->edebug("Subject: {$subject}"); - $this->edebug("Headers: {$header}"); - if (!$this->UseSendmailOptions || null === $params) { - $result = @mail($to, $subject, $body, $header); - } else { - $this->edebug("Additional params: {$params}"); - $result = @mail($to, $subject, $body, $header, $params); - } - $this->edebug('Result: ' . ($result ? 'true' : 'false')); - return $result; - } - - /** - * Output debugging info via a user-defined method. - * Only generates output if debug output is enabled. - * - * @see PHPMailer::$Debugoutput - * @see PHPMailer::$SMTPDebug - * - * @param string $str - */ - protected function edebug($str) - { - if ($this->SMTPDebug <= 0) { - return; - } - //Is this a PSR-3 logger? - if ($this->Debugoutput instanceof \Psr\Log\LoggerInterface) { - $this->Debugoutput->debug(rtrim($str, "\r\n")); - - return; - } - //Avoid clash with built-in function names - if (is_callable($this->Debugoutput) && !in_array($this->Debugoutput, ['error_log', 'html', 'echo'])) { - call_user_func($this->Debugoutput, $str, $this->SMTPDebug); - - return; - } - switch ($this->Debugoutput) { - case 'error_log': - //Don't output, just log - /** @noinspection ForgottenDebugOutputInspection */ - error_log($str); - break; - case 'html': - //Cleans up output a bit for a better looking, HTML-safe output - echo htmlentities( - preg_replace('/[\r\n]+/', '', $str), - ENT_QUOTES, - 'UTF-8' - ), "
\n"; - break; - case 'echo': - default: - //Normalize line breaks - $str = preg_replace('/\r\n|\r/m', "\n", $str); - echo gmdate('Y-m-d H:i:s'), - "\t", - //Trim trailing space - trim( - //Indent for readability, except for trailing break - str_replace( - "\n", - "\n \t ", - trim($str) - ) - ), - "\n"; - } - } - - /** - * Sets message type to HTML or plain. - * - * @param bool $isHtml True for HTML mode - */ - public function isHTML($isHtml = true) - { - if ($isHtml) { - $this->ContentType = static::CONTENT_TYPE_TEXT_HTML; - } else { - $this->ContentType = static::CONTENT_TYPE_PLAINTEXT; - } - } - - /** - * Send messages using SMTP. - */ - public function isSMTP() - { - $this->Mailer = 'smtp'; - } - - /** - * Send messages using PHP's mail() function. - */ - public function isMail() - { - $this->Mailer = 'mail'; - } - - /** - * Send messages using $Sendmail. - */ - public function isSendmail() - { - $ini_sendmail_path = ini_get('sendmail_path'); - - if (false === stripos($ini_sendmail_path, 'sendmail')) { - $this->Sendmail = '/usr/sbin/sendmail'; - } else { - $this->Sendmail = $ini_sendmail_path; - } - $this->Mailer = 'sendmail'; - } - - /** - * Send messages using qmail. - */ - public function isQmail() - { - $ini_sendmail_path = ini_get('sendmail_path'); - - if (false === stripos($ini_sendmail_path, 'qmail')) { - $this->Sendmail = '/var/qmail/bin/qmail-inject'; - } else { - $this->Sendmail = $ini_sendmail_path; - } - $this->Mailer = 'qmail'; - } - - /** - * Add a "To" address. - * - * @param string $address The email address to send to - * @param string $name - * - * @throws Exception - * - * @return bool true on success, false if address already used or invalid in some way - */ - public function addAddress($address, $name = '') - { - return $this->addOrEnqueueAnAddress('to', $address, $name); - } - - /** - * Add a "CC" address. - * - * @param string $address The email address to send to - * @param string $name - * - * @throws Exception - * - * @return bool true on success, false if address already used or invalid in some way - */ - public function addCC($address, $name = '') - { - return $this->addOrEnqueueAnAddress('cc', $address, $name); - } - - /** - * Add a "BCC" address. - * - * @param string $address The email address to send to - * @param string $name - * - * @throws Exception - * - * @return bool true on success, false if address already used or invalid in some way - */ - public function addBCC($address, $name = '') - { - return $this->addOrEnqueueAnAddress('bcc', $address, $name); - } - - /** - * Add a "Reply-To" address. - * - * @param string $address The email address to reply to - * @param string $name - * - * @throws Exception - * - * @return bool true on success, false if address already used or invalid in some way - */ - public function addReplyTo($address, $name = '') - { - return $this->addOrEnqueueAnAddress('Reply-To', $address, $name); - } - - /** - * Add an address to one of the recipient arrays or to the ReplyTo array. Because PHPMailer - * can't validate addresses with an IDN without knowing the PHPMailer::$CharSet (that can still - * be modified after calling this function), addition of such addresses is delayed until send(). - * Addresses that have been added already return false, but do not throw exceptions. - * - * @param string $kind One of 'to', 'cc', 'bcc', or 'Reply-To' - * @param string $address The email address - * @param string $name An optional username associated with the address - * - * @throws Exception - * - * @return bool true on success, false if address already used or invalid in some way - */ - protected function addOrEnqueueAnAddress($kind, $address, $name) - { - $pos = false; - if ($address !== null) { - $address = trim($address); - $pos = strrpos($address, '@'); - } - if (false === $pos) { - //At-sign is missing. - $error_message = sprintf( - '%s (%s): %s', - $this->lang('invalid_address'), - $kind, - $address - ); - $this->setError($error_message); - $this->edebug($error_message); - if ($this->exceptions) { - throw new Exception($error_message); - } - - return false; - } - if ($name !== null && is_string($name)) { - $name = trim(preg_replace('/[\r\n]+/', '', $name)); //Strip breaks and trim - } else { - $name = ''; - } - $params = [$kind, $address, $name]; - //Enqueue addresses with IDN until we know the PHPMailer::$CharSet. - //Domain is assumed to be whatever is after the last @ symbol in the address - if (static::idnSupported() && $this->has8bitChars(substr($address, ++$pos))) { - if ('Reply-To' !== $kind) { - if (!array_key_exists($address, $this->RecipientsQueue)) { - $this->RecipientsQueue[$address] = $params; - - return true; - } - } elseif (!array_key_exists($address, $this->ReplyToQueue)) { - $this->ReplyToQueue[$address] = $params; - - return true; - } - - return false; - } - - //Immediately add standard addresses without IDN. - return call_user_func_array([$this, 'addAnAddress'], $params); - } - - /** - * Set the boundaries to use for delimiting MIME parts. - * If you override this, ensure you set all 3 boundaries to unique values. - * The default boundaries include a "=_" sequence which cannot occur in quoted-printable bodies, - * as suggested by https://www.rfc-editor.org/rfc/rfc2045#section-6.7 - * - * @return void - */ - public function setBoundaries() - { - $this->uniqueid = $this->generateId(); - $this->boundary[1] = 'b1=_' . $this->uniqueid; - $this->boundary[2] = 'b2=_' . $this->uniqueid; - $this->boundary[3] = 'b3=_' . $this->uniqueid; - } - - /** - * Add an address to one of the recipient arrays or to the ReplyTo array. - * Addresses that have been added already return false, but do not throw exceptions. - * - * @param string $kind One of 'to', 'cc', 'bcc', or 'ReplyTo' - * @param string $address The email address to send, resp. to reply to - * @param string $name - * - * @throws Exception - * - * @return bool true on success, false if address already used or invalid in some way - */ - protected function addAnAddress($kind, $address, $name = '') - { - if (!in_array($kind, ['to', 'cc', 'bcc', 'Reply-To'])) { - $error_message = sprintf( - '%s: %s', - $this->lang('Invalid recipient kind'), - $kind - ); - $this->setError($error_message); - $this->edebug($error_message); - if ($this->exceptions) { - throw new Exception($error_message); - } - - return false; - } - if (!static::validateAddress($address)) { - $error_message = sprintf( - '%s (%s): %s', - $this->lang('invalid_address'), - $kind, - $address - ); - $this->setError($error_message); - $this->edebug($error_message); - if ($this->exceptions) { - throw new Exception($error_message); - } - - return false; - } - if ('Reply-To' !== $kind) { - if (!array_key_exists(strtolower($address), $this->all_recipients)) { - $this->{$kind}[] = [$address, $name]; - $this->all_recipients[strtolower($address)] = true; - - return true; - } - } elseif (!array_key_exists(strtolower($address), $this->ReplyTo)) { - $this->ReplyTo[strtolower($address)] = [$address, $name]; - - return true; - } - - return false; - } - - /** - * Parse and validate a string containing one or more RFC822-style comma-separated email addresses - * of the form "display name
" into an array of name/address pairs. - * Uses the imap_rfc822_parse_adrlist function if the IMAP extension is available. - * Note that quotes in the name part are removed. - * - * @see https://www.andrew.cmu.edu/user/agreen1/testing/mrbs/web/Mail/RFC822.php A more careful implementation - * - * @param string $addrstr The address list string - * @param bool $useimap Whether to use the IMAP extension to parse the list - * @param string $charset The charset to use when decoding the address list string. - * - * @return array - */ - public static function parseAddresses($addrstr, $useimap = true, $charset = self::CHARSET_ISO88591) - { - $addresses = []; - if ($useimap && function_exists('imap_rfc822_parse_adrlist')) { - //Use this built-in parser if it's available - $list = imap_rfc822_parse_adrlist($addrstr, ''); - // Clear any potential IMAP errors to get rid of notices being thrown at end of script. - imap_errors(); - foreach ($list as $address) { - if ( - '.SYNTAX-ERROR.' !== $address->host && - static::validateAddress($address->mailbox . '@' . $address->host) - ) { - //Decode the name part if it's present and encoded - if ( - property_exists($address, 'personal') && - //Check for a Mbstring constant rather than using extension_loaded, which is sometimes disabled - defined('MB_CASE_UPPER') && - preg_match('/^=\?.*\?=$/s', $address->personal) - ) { - $origCharset = mb_internal_encoding(); - mb_internal_encoding($charset); - //Undo any RFC2047-encoded spaces-as-underscores - $address->personal = str_replace('_', '=20', $address->personal); - //Decode the name - $address->personal = mb_decode_mimeheader($address->personal); - mb_internal_encoding($origCharset); - } - - $addresses[] = [ - 'name' => (property_exists($address, 'personal') ? $address->personal : ''), - 'address' => $address->mailbox . '@' . $address->host, - ]; - } - } - } else { - //Use this simpler parser - $list = explode(',', $addrstr); - foreach ($list as $address) { - $address = trim($address); - //Is there a separate name part? - if (strpos($address, '<') === false) { - //No separate name, just use the whole thing - if (static::validateAddress($address)) { - $addresses[] = [ - 'name' => '', - 'address' => $address, - ]; - } - } else { - list($name, $email) = explode('<', $address); - $email = trim(str_replace('>', '', $email)); - $name = trim($name); - if (static::validateAddress($email)) { - //Check for a Mbstring constant rather than using extension_loaded, which is sometimes disabled - //If this name is encoded, decode it - if (defined('MB_CASE_UPPER') && preg_match('/^=\?.*\?=$/s', $name)) { - $origCharset = mb_internal_encoding(); - mb_internal_encoding($charset); - //Undo any RFC2047-encoded spaces-as-underscores - $name = str_replace('_', '=20', $name); - //Decode the name - $name = mb_decode_mimeheader($name); - mb_internal_encoding($origCharset); - } - $addresses[] = [ - //Remove any surrounding quotes and spaces from the name - 'name' => trim($name, '\'" '), - 'address' => $email, - ]; - } - } - } - } - - return $addresses; - } - - /** - * Set the From and FromName properties. - * - * @param string $address - * @param string $name - * @param bool $auto Whether to also set the Sender address, defaults to true - * - * @throws Exception - * - * @return bool - */ - public function setFrom($address, $name = '', $auto = true) - { - $address = trim((string)$address); - $name = trim(preg_replace('/[\r\n]+/', '', $name)); //Strip breaks and trim - //Don't validate now addresses with IDN. Will be done in send(). - $pos = strrpos($address, '@'); - if ( - (false === $pos) - || ((!$this->has8bitChars(substr($address, ++$pos)) || !static::idnSupported()) - && !static::validateAddress($address)) - ) { - $error_message = sprintf( - '%s (From): %s', - $this->lang('invalid_address'), - $address - ); - $this->setError($error_message); - $this->edebug($error_message); - if ($this->exceptions) { - throw new Exception($error_message); - } - - return false; - } - $this->From = $address; - $this->FromName = $name; - if ($auto && empty($this->Sender)) { - $this->Sender = $address; - } - - return true; - } - - /** - * Return the Message-ID header of the last email. - * Technically this is the value from the last time the headers were created, - * but it's also the message ID of the last sent message except in - * pathological cases. - * - * @return string - */ - public function getLastMessageID() - { - return $this->lastMessageID; - } - - /** - * Check that a string looks like an email address. - * Validation patterns supported: - * * `auto` Pick best pattern automatically; - * * `pcre8` Use the squiloople.com pattern, requires PCRE > 8.0; - * * `pcre` Use old PCRE implementation; - * * `php` Use PHP built-in FILTER_VALIDATE_EMAIL; - * * `html5` Use the pattern given by the HTML5 spec for 'email' type form input elements. - * * `noregex` Don't use a regex: super fast, really dumb. - * Alternatively you may pass in a callable to inject your own validator, for example: - * - * ```php - * PHPMailer::validateAddress('user@example.com', function($address) { - * return (strpos($address, '@') !== false); - * }); - * ``` - * - * You can also set the PHPMailer::$validator static to a callable, allowing built-in methods to use your validator. - * - * @param string $address The email address to check - * @param string|callable $patternselect Which pattern to use - * - * @return bool - */ - public static function validateAddress($address, $patternselect = null) - { - if (null === $patternselect) { - $patternselect = static::$validator; - } - //Don't allow strings as callables, see SECURITY.md and CVE-2021-3603 - if (is_callable($patternselect) && !is_string($patternselect)) { - return call_user_func($patternselect, $address); - } - //Reject line breaks in addresses; it's valid RFC5322, but not RFC5321 - if (strpos($address, "\n") !== false || strpos($address, "\r") !== false) { - return false; - } - switch ($patternselect) { - case 'pcre': //Kept for BC - case 'pcre8': - /* - * A more complex and more permissive version of the RFC5322 regex on which FILTER_VALIDATE_EMAIL - * is based. - * In addition to the addresses allowed by filter_var, also permits: - * * dotless domains: `a@b` - * * comments: `1234 @ local(blah) .machine .example` - * * quoted elements: `'"test blah"@example.org'` - * * numeric TLDs: `a@b.123` - * * unbracketed IPv4 literals: `a@192.168.0.1` - * * IPv6 literals: 'first.last@[IPv6:a1::]' - * Not all of these will necessarily work for sending! - * - * @copyright 2009-2010 Michael Rushton - * Feel free to use and redistribute this code. But please keep this copyright notice. - */ - return (bool) preg_match( - '/^(?!(?>(?1)"?(?>\\\[ -~]|[^"])"?(?1)){255,})(?!(?>(?1)"?(?>\\\[ -~]|[^"])"?(?1)){65,}@)' . - '((?>(?>(?>((?>(?>(?>\x0D\x0A)?[\t ])+|(?>[\t ]*\x0D\x0A)?[\t ]+)?)(\((?>(?2)' . - '(?>[\x01-\x08\x0B\x0C\x0E-\'*-\[\]-\x7F]|\\\[\x00-\x7F]|(?3)))*(?2)\)))+(?2))|(?2))?)' . - '([!#-\'*+\/-9=?^-~-]+|"(?>(?2)(?>[\x01-\x08\x0B\x0C\x0E-!#-\[\]-\x7F]|\\\[\x00-\x7F]))*' . - '(?2)")(?>(?1)\.(?1)(?4))*(?1)@(?!(?1)[a-z0-9-]{64,})(?1)(?>([a-z0-9](?>[a-z0-9-]*[a-z0-9])?)' . - '(?>(?1)\.(?!(?1)[a-z0-9-]{64,})(?1)(?5)){0,126}|\[(?:(?>IPv6:(?>([a-f0-9]{1,4})(?>:(?6)){7}' . - '|(?!(?:.*[a-f0-9][:\]]){8,})((?6)(?>:(?6)){0,6})?::(?7)?))|(?>(?>IPv6:(?>(?6)(?>:(?6)){5}:' . - '|(?!(?:.*[a-f0-9]:){6,})(?8)?::(?>((?6)(?>:(?6)){0,4}):)?))?(25[0-5]|2[0-4][0-9]|1[0-9]{2}' . - '|[1-9]?[0-9])(?>\.(?9)){3}))\])(?1)$/isD', - $address - ); - case 'html5': - /* - * This is the pattern used in the HTML5 spec for validation of 'email' type form input elements. - * - * @see https://html.spec.whatwg.org/#e-mail-state-(type=email) - */ - return (bool) preg_match( - '/^[a-zA-Z0-9.!#$%&\'*+\/=?^_`{|}~-]+@[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}' . - '[a-zA-Z0-9])?(?:\.[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?)*$/sD', - $address - ); - case 'php': - default: - return filter_var($address, FILTER_VALIDATE_EMAIL) !== false; - } - } - - /** - * Tells whether IDNs (Internationalized Domain Names) are supported or not. This requires the - * `intl` and `mbstring` PHP extensions. - * - * @return bool `true` if required functions for IDN support are present - */ - public static function idnSupported() - { - return function_exists('idn_to_ascii') && function_exists('mb_convert_encoding'); - } - - /** - * Converts IDN in given email address to its ASCII form, also known as punycode, if possible. - * Important: Address must be passed in same encoding as currently set in PHPMailer::$CharSet. - * This function silently returns unmodified address if: - * - No conversion is necessary (i.e. domain name is not an IDN, or is already in ASCII form) - * - Conversion to punycode is impossible (e.g. required PHP functions are not available) - * or fails for any reason (e.g. domain contains characters not allowed in an IDN). - * - * @see PHPMailer::$CharSet - * - * @param string $address The email address to convert - * - * @return string The encoded address in ASCII form - */ - public function punyencodeAddress($address) - { - //Verify we have required functions, CharSet, and at-sign. - $pos = strrpos($address, '@'); - if ( - !empty($this->CharSet) && - false !== $pos && - static::idnSupported() - ) { - $domain = substr($address, ++$pos); - //Verify CharSet string is a valid one, and domain properly encoded in this CharSet. - if ($this->has8bitChars($domain) && @mb_check_encoding($domain, $this->CharSet)) { - //Convert the domain from whatever charset it's in to UTF-8 - $domain = mb_convert_encoding($domain, self::CHARSET_UTF8, $this->CharSet); - //Ignore IDE complaints about this line - method signature changed in PHP 5.4 - $errorcode = 0; - if (defined('INTL_IDNA_VARIANT_UTS46')) { - //Use the current punycode standard (appeared in PHP 7.2) - $punycode = idn_to_ascii( - $domain, - \IDNA_DEFAULT | \IDNA_USE_STD3_RULES | \IDNA_CHECK_BIDI | - \IDNA_CHECK_CONTEXTJ | \IDNA_NONTRANSITIONAL_TO_ASCII, - \INTL_IDNA_VARIANT_UTS46 - ); - } elseif (defined('INTL_IDNA_VARIANT_2003')) { - //Fall back to this old, deprecated/removed encoding - // phpcs:ignore PHPCompatibility.Constants.RemovedConstants.intl_idna_variant_2003Deprecated - $punycode = idn_to_ascii($domain, $errorcode, \INTL_IDNA_VARIANT_2003); - } else { - //Fall back to a default we don't know about - // phpcs:ignore PHPCompatibility.ParameterValues.NewIDNVariantDefault.NotSet - $punycode = idn_to_ascii($domain, $errorcode); - } - if (false !== $punycode) { - return substr($address, 0, $pos) . $punycode; - } - } - } - - return $address; - } - - /** - * Create a message and send it. - * Uses the sending method specified by $Mailer. - * - * @throws Exception - * - * @return bool false on error - See the ErrorInfo property for details of the error - */ - public function send() - { - try { - if (!$this->preSend()) { - return false; - } - - return $this->postSend(); - } catch (Exception $exc) { - $this->mailHeader = ''; - $this->setError($exc->getMessage()); - if ($this->exceptions) { - throw $exc; - } - - return false; - } - } - - /** - * Prepare a message for sending. - * - * @throws Exception - * - * @return bool - */ - public function preSend() - { - if ( - 'smtp' === $this->Mailer - || ('mail' === $this->Mailer && (\PHP_VERSION_ID >= 80000 || stripos(PHP_OS, 'WIN') === 0)) - ) { - //SMTP mandates RFC-compliant line endings - //and it's also used with mail() on Windows - static::setLE(self::CRLF); - } else { - //Maintain backward compatibility with legacy Linux command line mailers - static::setLE(PHP_EOL); - } - //Check for buggy PHP versions that add a header with an incorrect line break - if ( - 'mail' === $this->Mailer - && ((\PHP_VERSION_ID >= 70000 && \PHP_VERSION_ID < 70017) - || (\PHP_VERSION_ID >= 70100 && \PHP_VERSION_ID < 70103)) - && ini_get('mail.add_x_header') === '1' - && stripos(PHP_OS, 'WIN') === 0 - ) { - trigger_error($this->lang('buggy_php'), E_USER_WARNING); - } - - try { - $this->error_count = 0; //Reset errors - $this->mailHeader = ''; - - //Dequeue recipient and Reply-To addresses with IDN - foreach (array_merge($this->RecipientsQueue, $this->ReplyToQueue) as $params) { - $params[1] = $this->punyencodeAddress($params[1]); - call_user_func_array([$this, 'addAnAddress'], $params); - } - if (count($this->to) + count($this->cc) + count($this->bcc) < 1) { - throw new Exception($this->lang('provide_address'), self::STOP_CRITICAL); - } - - //Validate From, Sender, and ConfirmReadingTo addresses - foreach (['From', 'Sender', 'ConfirmReadingTo'] as $address_kind) { - if ($this->{$address_kind} === null) { - $this->{$address_kind} = ''; - continue; - } - $this->{$address_kind} = trim($this->{$address_kind}); - if (empty($this->{$address_kind})) { - continue; - } - $this->{$address_kind} = $this->punyencodeAddress($this->{$address_kind}); - if (!static::validateAddress($this->{$address_kind})) { - $error_message = sprintf( - '%s (%s): %s', - $this->lang('invalid_address'), - $address_kind, - $this->{$address_kind} - ); - $this->setError($error_message); - $this->edebug($error_message); - if ($this->exceptions) { - throw new Exception($error_message); - } - - return false; - } - } - - //Set whether the message is multipart/alternative - if ($this->alternativeExists()) { - $this->ContentType = static::CONTENT_TYPE_MULTIPART_ALTERNATIVE; - } - - $this->setMessageType(); - //Refuse to send an empty message unless we are specifically allowing it - if (!$this->AllowEmpty && empty($this->Body)) { - throw new Exception($this->lang('empty_message'), self::STOP_CRITICAL); - } - - //Trim subject consistently - $this->Subject = trim($this->Subject); - //Create body before headers in case body makes changes to headers (e.g. altering transfer encoding) - $this->MIMEHeader = ''; - $this->MIMEBody = $this->createBody(); - //createBody may have added some headers, so retain them - $tempheaders = $this->MIMEHeader; - $this->MIMEHeader = $this->createHeader(); - $this->MIMEHeader .= $tempheaders; - - //To capture the complete message when using mail(), create - //an extra header list which createHeader() doesn't fold in - if ('mail' === $this->Mailer) { - if (count($this->to) > 0) { - $this->mailHeader .= $this->addrAppend('To', $this->to); - } else { - $this->mailHeader .= $this->headerLine('To', 'undisclosed-recipients:;'); - } - $this->mailHeader .= $this->headerLine( - 'Subject', - $this->encodeHeader($this->secureHeader($this->Subject)) - ); - } - - //Sign with DKIM if enabled - if ( - !empty($this->DKIM_domain) - && !empty($this->DKIM_selector) - && (!empty($this->DKIM_private_string) - || (!empty($this->DKIM_private) - && static::isPermittedPath($this->DKIM_private) - && file_exists($this->DKIM_private) - ) - ) - ) { - $header_dkim = $this->DKIM_Add( - $this->MIMEHeader . $this->mailHeader, - $this->encodeHeader($this->secureHeader($this->Subject)), - $this->MIMEBody - ); - $this->MIMEHeader = static::stripTrailingWSP($this->MIMEHeader) . static::$LE . - static::normalizeBreaks($header_dkim) . static::$LE; - } - - return true; - } catch (Exception $exc) { - $this->setError($exc->getMessage()); - if ($this->exceptions) { - throw $exc; - } - - return false; - } - } - - /** - * Actually send a message via the selected mechanism. - * - * @throws Exception - * - * @return bool - */ - public function postSend() - { - try { - //Choose the mailer and send through it - switch ($this->Mailer) { - case 'sendmail': - case 'qmail': - return $this->sendmailSend($this->MIMEHeader, $this->MIMEBody); - case 'smtp': - return $this->smtpSend($this->MIMEHeader, $this->MIMEBody); - case 'mail': - return $this->mailSend($this->MIMEHeader, $this->MIMEBody); - default: - $sendMethod = $this->Mailer . 'Send'; - if (method_exists($this, $sendMethod)) { - return $this->{$sendMethod}($this->MIMEHeader, $this->MIMEBody); - } - - return $this->mailSend($this->MIMEHeader, $this->MIMEBody); - } - } catch (Exception $exc) { - $this->setError($exc->getMessage()); - $this->edebug($exc->getMessage()); - if ($this->Mailer === 'smtp' && $this->SMTPKeepAlive == true && $this->smtp->connected()) { - $this->smtp->reset(); - } - if ($this->exceptions) { - throw $exc; - } - } - - return false; - } - - /** - * Send mail using the $Sendmail program. - * - * @see PHPMailer::$Sendmail - * - * @param string $header The message headers - * @param string $body The message body - * - * @throws Exception - * - * @return bool - */ - protected function sendmailSend($header, $body) - { - if ($this->Mailer === 'qmail') { - $this->edebug('Sending with qmail'); - } else { - $this->edebug('Sending with sendmail'); - } - $header = static::stripTrailingWSP($header) . static::$LE . static::$LE; - //This sets the SMTP envelope sender which gets turned into a return-path header by the receiver - //A space after `-f` is optional, but there is a long history of its presence - //causing problems, so we don't use one - //Exim docs: https://www.exim.org/exim-html-current/doc/html/spec_html/ch-the_exim_command_line.html - //Sendmail docs: https://www.sendmail.org/~ca/email/man/sendmail.html - //Example problem: https://www.drupal.org/node/1057954 - - //PHP 5.6 workaround - $sendmail_from_value = ini_get('sendmail_from'); - if (empty($this->Sender) && !empty($sendmail_from_value)) { - //PHP config has a sender address we can use - $this->Sender = ini_get('sendmail_from'); - } - //CVE-2016-10033, CVE-2016-10045: Don't pass -f if characters will be escaped. - if (!empty($this->Sender) && static::validateAddress($this->Sender) && self::isShellSafe($this->Sender)) { - if ($this->Mailer === 'qmail') { - $sendmailFmt = '%s -f%s'; - } else { - $sendmailFmt = '%s -oi -f%s -t'; - } - } else { - //allow sendmail to choose a default envelope sender. It may - //seem preferable to force it to use the From header as with - //SMTP, but that introduces new problems (see - //), and - //it has historically worked this way. - $sendmailFmt = '%s -oi -t'; - } - - $sendmail = sprintf($sendmailFmt, escapeshellcmd($this->Sendmail), $this->Sender); - $this->edebug('Sendmail path: ' . $this->Sendmail); - $this->edebug('Sendmail command: ' . $sendmail); - $this->edebug('Envelope sender: ' . $this->Sender); - $this->edebug("Headers: {$header}"); - - if ($this->SingleTo) { - foreach ($this->SingleToArray as $toAddr) { - $mail = @popen($sendmail, 'w'); - if (!$mail) { - throw new Exception($this->lang('execute') . $this->Sendmail, self::STOP_CRITICAL); - } - $this->edebug("To: {$toAddr}"); - fwrite($mail, 'To: ' . $toAddr . "\n"); - fwrite($mail, $header); - fwrite($mail, $body); - $result = pclose($mail); - $addrinfo = static::parseAddresses($toAddr, true, $this->CharSet); - $this->doCallback( - ($result === 0), - [[$addrinfo['address'], $addrinfo['name']]], - $this->cc, - $this->bcc, - $this->Subject, - $body, - $this->From, - [] - ); - $this->edebug("Result: " . ($result === 0 ? 'true' : 'false')); - if (0 !== $result) { - throw new Exception($this->lang('execute') . $this->Sendmail, self::STOP_CRITICAL); - } - } - } else { - $mail = @popen($sendmail, 'w'); - if (!$mail) { - throw new Exception($this->lang('execute') . $this->Sendmail, self::STOP_CRITICAL); - } - fwrite($mail, $header); - fwrite($mail, $body); - $result = pclose($mail); - $this->doCallback( - ($result === 0), - $this->to, - $this->cc, - $this->bcc, - $this->Subject, - $body, - $this->From, - [] - ); - $this->edebug("Result: " . ($result === 0 ? 'true' : 'false')); - if (0 !== $result) { - throw new Exception($this->lang('execute') . $this->Sendmail, self::STOP_CRITICAL); - } - } - - return true; - } - - /** - * Fix CVE-2016-10033 and CVE-2016-10045 by disallowing potentially unsafe shell characters. - * Note that escapeshellarg and escapeshellcmd are inadequate for our purposes, especially on Windows. - * - * @see https://github.com/PHPMailer/PHPMailer/issues/924 CVE-2016-10045 bug report - * - * @param string $string The string to be validated - * - * @return bool - */ - protected static function isShellSafe($string) - { - //It's not possible to use shell commands safely (which includes the mail() function) without escapeshellarg, - //but some hosting providers disable it, creating a security problem that we don't want to have to deal with, - //so we don't. - if (!function_exists('escapeshellarg') || !function_exists('escapeshellcmd')) { - return false; - } - - if ( - escapeshellcmd($string) !== $string - || !in_array(escapeshellarg($string), ["'$string'", "\"$string\""]) - ) { - return false; - } - - $length = strlen($string); - - for ($i = 0; $i < $length; ++$i) { - $c = $string[$i]; - - //All other characters have a special meaning in at least one common shell, including = and +. - //Full stop (.) has a special meaning in cmd.exe, but its impact should be negligible here. - //Note that this does permit non-Latin alphanumeric characters based on the current locale. - if (!ctype_alnum($c) && strpos('@_-.', $c) === false) { - return false; - } - } - - return true; - } - - /** - * Check whether a file path is of a permitted type. - * Used to reject URLs and phar files from functions that access local file paths, - * such as addAttachment. - * - * @param string $path A relative or absolute path to a file - * - * @return bool - */ - protected static function isPermittedPath($path) - { - //Matches scheme definition from https://www.rfc-editor.org/rfc/rfc3986#section-3.1 - return !preg_match('#^[a-z][a-z\d+.-]*://#i', $path); - } - - /** - * Check whether a file path is safe, accessible, and readable. - * - * @param string $path A relative or absolute path to a file - * - * @return bool - */ - protected static function fileIsAccessible($path) - { - if (!static::isPermittedPath($path)) { - return false; - } - $readable = is_file($path); - //If not a UNC path (expected to start with \\), check read permission, see #2069 - if (strpos($path, '\\\\') !== 0) { - $readable = $readable && is_readable($path); - } - return $readable; - } - - /** - * Send mail using the PHP mail() function. - * - * @see https://www.php.net/manual/en/book.mail.php - * - * @param string $header The message headers - * @param string $body The message body - * - * @throws Exception - * - * @return bool - */ - protected function mailSend($header, $body) - { - $header = static::stripTrailingWSP($header) . static::$LE . static::$LE; - - $toArr = []; - foreach ($this->to as $toaddr) { - $toArr[] = $this->addrFormat($toaddr); - } - $to = trim(implode(', ', $toArr)); - - //If there are no To-addresses (e.g. when sending only to BCC-addresses) - //the following should be added to get a correct DKIM-signature. - //Compare with $this->preSend() - if ($to === '') { - $to = 'undisclosed-recipients:;'; - } - - $params = null; - //This sets the SMTP envelope sender which gets turned into a return-path header by the receiver - //A space after `-f` is optional, but there is a long history of its presence - //causing problems, so we don't use one - //Exim docs: https://www.exim.org/exim-html-current/doc/html/spec_html/ch-the_exim_command_line.html - //Sendmail docs: https://www.sendmail.org/~ca/email/man/sendmail.html - //Example problem: https://www.drupal.org/node/1057954 - //CVE-2016-10033, CVE-2016-10045: Don't pass -f if characters will be escaped. - - //PHP 5.6 workaround - $sendmail_from_value = ini_get('sendmail_from'); - if (empty($this->Sender) && !empty($sendmail_from_value)) { - //PHP config has a sender address we can use - $this->Sender = ini_get('sendmail_from'); - } - if (!empty($this->Sender) && static::validateAddress($this->Sender)) { - if (self::isShellSafe($this->Sender)) { - $params = sprintf('-f%s', $this->Sender); - } - $old_from = ini_get('sendmail_from'); - ini_set('sendmail_from', $this->Sender); - } - $result = false; - if ($this->SingleTo && count($toArr) > 1) { - foreach ($toArr as $toAddr) { - $result = $this->mailPassthru($toAddr, $this->Subject, $body, $header, $params); - $addrinfo = static::parseAddresses($toAddr, true, $this->CharSet); - $this->doCallback( - $result, - [[$addrinfo['address'], $addrinfo['name']]], - $this->cc, - $this->bcc, - $this->Subject, - $body, - $this->From, - [] - ); - } - } else { - $result = $this->mailPassthru($to, $this->Subject, $body, $header, $params); - $this->doCallback($result, $this->to, $this->cc, $this->bcc, $this->Subject, $body, $this->From, []); - } - if (isset($old_from)) { - ini_set('sendmail_from', $old_from); - } - if (!$result) { - throw new Exception($this->lang('instantiate'), self::STOP_CRITICAL); - } - - return true; - } - - /** - * Get an instance to use for SMTP operations. - * Override this function to load your own SMTP implementation, - * or set one with setSMTPInstance. - * - * @return SMTP - */ - public function getSMTPInstance() - { - if (!is_object($this->smtp)) { - $this->smtp = new SMTP(); - } - - return $this->smtp; - } - - /** - * Provide an instance to use for SMTP operations. - * - * @return SMTP - */ - public function setSMTPInstance(SMTP $smtp) - { - $this->smtp = $smtp; - - return $this->smtp; - } - - /** - * Provide SMTP XCLIENT attributes - * - * @param string $name Attribute name - * @param ?string $value Attribute value - * - * @return bool - */ - public function setSMTPXclientAttribute($name, $value) - { - if (!in_array($name, SMTP::$xclient_allowed_attributes)) { - return false; - } - if (isset($this->SMTPXClient[$name]) && $value === null) { - unset($this->SMTPXClient[$name]); - } elseif ($value !== null) { - $this->SMTPXClient[$name] = $value; - } - - return true; - } - - /** - * Get SMTP XCLIENT attributes - * - * @return array - */ - public function getSMTPXclientAttributes() - { - return $this->SMTPXClient; - } - - /** - * Send mail via SMTP. - * Returns false if there is a bad MAIL FROM, RCPT, or DATA input. - * - * @see PHPMailer::setSMTPInstance() to use a different class. - * - * @uses \PHPMailer\PHPMailer\SMTP - * - * @param string $header The message headers - * @param string $body The message body - * - * @throws Exception - * - * @return bool - */ - protected function smtpSend($header, $body) - { - $header = static::stripTrailingWSP($header) . static::$LE . static::$LE; - $bad_rcpt = []; - if (!$this->smtpConnect($this->SMTPOptions)) { - throw new Exception($this->lang('smtp_connect_failed'), self::STOP_CRITICAL); - } - //Sender already validated in preSend() - if ('' === $this->Sender) { - $smtp_from = $this->From; - } else { - $smtp_from = $this->Sender; - } - if (count($this->SMTPXClient)) { - $this->smtp->xclient($this->SMTPXClient); - } - if (!$this->smtp->mail($smtp_from)) { - $this->setError($this->lang('from_failed') . $smtp_from . ' : ' . implode(',', $this->smtp->getError())); - throw new Exception($this->ErrorInfo, self::STOP_CRITICAL); - } - - $callbacks = []; - //Attempt to send to all recipients - foreach ([$this->to, $this->cc, $this->bcc] as $togroup) { - foreach ($togroup as $to) { - if (!$this->smtp->recipient($to[0], $this->dsn)) { - $error = $this->smtp->getError(); - $bad_rcpt[] = ['to' => $to[0], 'error' => $error['detail']]; - $isSent = false; - } else { - $isSent = true; - } - - $callbacks[] = ['issent' => $isSent, 'to' => $to[0], 'name' => $to[1]]; - } - } - - //Only send the DATA command if we have viable recipients - if ((count($this->all_recipients) > count($bad_rcpt)) && !$this->smtp->data($header . $body)) { - throw new Exception($this->lang('data_not_accepted'), self::STOP_CRITICAL); - } - - $smtp_transaction_id = $this->smtp->getLastTransactionID(); - - if ($this->SMTPKeepAlive) { - $this->smtp->reset(); - } else { - $this->smtp->quit(); - $this->smtp->close(); - } - - foreach ($callbacks as $cb) { - $this->doCallback( - $cb['issent'], - [[$cb['to'], $cb['name']]], - [], - [], - $this->Subject, - $body, - $this->From, - ['smtp_transaction_id' => $smtp_transaction_id] - ); - } - - //Create error message for any bad addresses - if (count($bad_rcpt) > 0) { - $errstr = ''; - foreach ($bad_rcpt as $bad) { - $errstr .= $bad['to'] . ': ' . $bad['error']; - } - throw new Exception($this->lang('recipients_failed') . $errstr, self::STOP_CONTINUE); - } - - return true; - } - - /** - * Initiate a connection to an SMTP server. - * Returns false if the operation failed. - * - * @param array $options An array of options compatible with stream_context_create() - * - * @throws Exception - * - * @uses \PHPMailer\PHPMailer\SMTP - * - * @return bool - */ - public function smtpConnect($options = null) - { - if (null === $this->smtp) { - $this->smtp = $this->getSMTPInstance(); - } - - //If no options are provided, use whatever is set in the instance - if (null === $options) { - $options = $this->SMTPOptions; - } - - //Already connected? - if ($this->smtp->connected()) { - return true; - } - - $this->smtp->setTimeout($this->Timeout); - $this->smtp->setDebugLevel($this->SMTPDebug); - $this->smtp->setDebugOutput($this->Debugoutput); - $this->smtp->setVerp($this->do_verp); - if ($this->Host === null) { - $this->Host = 'localhost'; - } - $hosts = explode(';', $this->Host); - $lastexception = null; - - foreach ($hosts as $hostentry) { - $hostinfo = []; - if ( - !preg_match( - '/^(?:(ssl|tls):\/\/)?(.+?)(?::(\d+))?$/', - trim($hostentry), - $hostinfo - ) - ) { - $this->edebug($this->lang('invalid_hostentry') . ' ' . trim($hostentry)); - //Not a valid host entry - continue; - } - //$hostinfo[1]: optional ssl or tls prefix - //$hostinfo[2]: the hostname - //$hostinfo[3]: optional port number - //The host string prefix can temporarily override the current setting for SMTPSecure - //If it's not specified, the default value is used - - //Check the host name is a valid name or IP address before trying to use it - if (!static::isValidHost($hostinfo[2])) { - $this->edebug($this->lang('invalid_host') . ' ' . $hostinfo[2]); - continue; - } - $prefix = ''; - $secure = $this->SMTPSecure; - $tls = (static::ENCRYPTION_STARTTLS === $this->SMTPSecure); - if ('ssl' === $hostinfo[1] || ('' === $hostinfo[1] && static::ENCRYPTION_SMTPS === $this->SMTPSecure)) { - $prefix = 'ssl://'; - $tls = false; //Can't have SSL and TLS at the same time - $secure = static::ENCRYPTION_SMTPS; - } elseif ('tls' === $hostinfo[1]) { - $tls = true; - //TLS doesn't use a prefix - $secure = static::ENCRYPTION_STARTTLS; - } - //Do we need the OpenSSL extension? - $sslext = defined('OPENSSL_ALGO_SHA256'); - if (static::ENCRYPTION_STARTTLS === $secure || static::ENCRYPTION_SMTPS === $secure) { - //Check for an OpenSSL constant rather than using extension_loaded, which is sometimes disabled - if (!$sslext) { - throw new Exception($this->lang('extension_missing') . 'openssl', self::STOP_CRITICAL); - } - } - $host = $hostinfo[2]; - $port = $this->Port; - if ( - array_key_exists(3, $hostinfo) && - is_numeric($hostinfo[3]) && - $hostinfo[3] > 0 && - $hostinfo[3] < 65536 - ) { - $port = (int) $hostinfo[3]; - } - if ($this->smtp->connect($prefix . $host, $port, $this->Timeout, $options)) { - try { - if ($this->Helo) { - $hello = $this->Helo; - } else { - $hello = $this->serverHostname(); - } - $this->smtp->hello($hello); - //Automatically enable TLS encryption if: - //* it's not disabled - //* we are not connecting to localhost - //* we have openssl extension - //* we are not already using SSL - //* the server offers STARTTLS - if ( - $this->SMTPAutoTLS && - $this->Host !== 'localhost' && - $sslext && - $secure !== 'ssl' && - $this->smtp->getServerExt('STARTTLS') - ) { - $tls = true; - } - if ($tls) { - if (!$this->smtp->startTLS()) { - $message = $this->getSmtpErrorMessage('connect_host'); - throw new Exception($message); - } - //We must resend EHLO after TLS negotiation - $this->smtp->hello($hello); - } - if ( - $this->SMTPAuth && !$this->smtp->authenticate( - $this->Username, - $this->Password, - $this->AuthType, - $this->oauth - ) - ) { - throw new Exception($this->lang('authenticate')); - } - - return true; - } catch (Exception $exc) { - $lastexception = $exc; - $this->edebug($exc->getMessage()); - //We must have connected, but then failed TLS or Auth, so close connection nicely - $this->smtp->quit(); - } - } - } - //If we get here, all connection attempts have failed, so close connection hard - $this->smtp->close(); - //As we've caught all exceptions, just report whatever the last one was - if ($this->exceptions && null !== $lastexception) { - throw $lastexception; - } - if ($this->exceptions) { - // no exception was thrown, likely $this->smtp->connect() failed - $message = $this->getSmtpErrorMessage('connect_host'); - throw new Exception($message); - } - - return false; - } - - /** - * Close the active SMTP session if one exists. - */ - public function smtpClose() - { - if ((null !== $this->smtp) && $this->smtp->connected()) { - $this->smtp->quit(); - $this->smtp->close(); - } - } - - /** - * Set the language for error messages. - * The default language is English. - * - * @param string $langcode ISO 639-1 2-character language code (e.g. French is "fr") - * Optionally, the language code can be enhanced with a 4-character - * script annotation and/or a 2-character country annotation. - * @param string $lang_path Path to the language file directory, with trailing separator (slash) - * Do not set this from user input! - * - * @return bool Returns true if the requested language was loaded, false otherwise. - */ - public function setLanguage($langcode = 'en', $lang_path = '') - { - //Backwards compatibility for renamed language codes - $renamed_langcodes = [ - 'br' => 'pt_br', - 'cz' => 'cs', - 'dk' => 'da', - 'no' => 'nb', - 'se' => 'sv', - 'rs' => 'sr', - 'tg' => 'tl', - 'am' => 'hy', - ]; - - if (array_key_exists($langcode, $renamed_langcodes)) { - $langcode = $renamed_langcodes[$langcode]; - } - - //Define full set of translatable strings in English - $PHPMAILER_LANG = [ - 'authenticate' => 'SMTP Error: Could not authenticate.', - 'buggy_php' => 'Your version of PHP is affected by a bug that may result in corrupted messages.' . - ' To fix it, switch to sending using SMTP, disable the mail.add_x_header option in' . - ' your php.ini, switch to MacOS or Linux, or upgrade your PHP to version 7.0.17+ or 7.1.3+.', - 'connect_host' => 'SMTP Error: Could not connect to SMTP host.', - 'data_not_accepted' => 'SMTP Error: data not accepted.', - 'empty_message' => 'Message body empty', - 'encoding' => 'Unknown encoding: ', - 'execute' => 'Could not execute: ', - 'extension_missing' => 'Extension missing: ', - 'file_access' => 'Could not access file: ', - 'file_open' => 'File Error: Could not open file: ', - 'from_failed' => 'The following From address failed: ', - 'instantiate' => 'Could not instantiate mail function.', - 'invalid_address' => 'Invalid address: ', - 'invalid_header' => 'Invalid header name or value', - 'invalid_hostentry' => 'Invalid hostentry: ', - 'invalid_host' => 'Invalid host: ', - 'mailer_not_supported' => ' mailer is not supported.', - 'provide_address' => 'You must provide at least one recipient email address.', - 'recipients_failed' => 'SMTP Error: The following recipients failed: ', - 'signing' => 'Signing Error: ', - 'smtp_code' => 'SMTP code: ', - 'smtp_code_ex' => 'Additional SMTP info: ', - 'smtp_connect_failed' => 'SMTP connect() failed.', - 'smtp_detail' => 'Detail: ', - 'smtp_error' => 'SMTP server error: ', - 'variable_set' => 'Cannot set or reset variable: ', - ]; - if (empty($lang_path)) { - //Calculate an absolute path so it can work if CWD is not here - $lang_path = dirname(__DIR__) . DIRECTORY_SEPARATOR . 'language' . DIRECTORY_SEPARATOR; - } - - //Validate $langcode - $foundlang = true; - $langcode = strtolower($langcode); - if ( - !preg_match('/^(?P[a-z]{2})(?P - - - \ No newline at end of file diff --git a/backup_2026-05-04/app/api/__init__.py b/backup_2026-05-04/app/api/__init__.py deleted file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..0000000000000000000000000000000000000000 diff --git a/backup_2026-05-04/app/api/schemas.py b/backup_2026-05-04/app/api/schemas.py deleted file mode 100644 index 3769f29cf891fb8935d2aff8e74c38c3bc4ebb2c..0000000000000000000000000000000000000000 --- a/backup_2026-05-04/app/api/schemas.py +++ /dev/null @@ -1,27 +0,0 @@ -from typing import List, Dict, Any - -from pydantic import BaseModel, Field - - -class ChatRequest(BaseModel): - message: str = Field(..., min_length=1) - history: List[Dict[str, str]] = Field(default_factory=list) - session_id: str | None = Field(default=None, description="Unique session identifier for chat persistence") - - -class ChatResponse(BaseModel): - reply: str - retrieved_chunks: List[Dict[str, Any]] = Field(default_factory=list) - - -class HealthResponse(BaseModel): - status: str - docs_loaded: bool - index_ready: bool - chunk_count: int - index_path: str - - -class SessionUpdateNameRequest(BaseModel): - session_id: str - user_name: str diff --git a/backup_2026-05-04/app/core/__init__.py b/backup_2026-05-04/app/core/__init__.py deleted file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..0000000000000000000000000000000000000000 diff --git a/backup_2026-05-04/app/core/config.py b/backup_2026-05-04/app/core/config.py deleted file mode 100644 index 96a368718de9c02e33bf2f9f8acc3b4403c2a111..0000000000000000000000000000000000000000 --- a/backup_2026-05-04/app/core/config.py +++ /dev/null @@ -1,51 +0,0 @@ -from functools import lru_cache -from pathlib import Path -from pydantic import Field, model_validator -from pydantic_settings import BaseSettings, SettingsConfigDict - - -class Settings(BaseSettings): - model_config = SettingsConfigDict(env_file=".env", env_file_encoding="utf-8") - - app_name: str = "Fast RAG Chatbot" - llm_provider: str = Field(default="groq", alias="LLM_PROVIDER") - groq_api_key: str = Field(default="", alias="GROQ_API_KEY") - groq_model: str = Field(default="qwen/qwen3-32b", alias="GROQ_MODEL") - groq_rewrite_model: str = Field(default="llama-3.1-8b-instant", alias="GROQ_REWRITE_MODEL") - hf_api_key: str = Field(default="", alias="HF_API_KEY") - hf_model: str = Field(default="meta-llama/Llama-3.1-8B-Instruct", alias="HF_MODEL") - embedding_model: str = Field(default="BAAI/bge-small-en-v1.5", alias="EMBEDDING_MODEL") - reranker_model: str = Field(default="BAAI/bge-reranker-base", alias="RERANKER_MODEL") - docs_dir: Path = Field(default=Path("docs"), alias="DOCS_DIR") - @model_validator(mode='after') - def set_persistent_paths(self) -> 'Settings': - # If we are on Hugging Face (detected by /data volume), - # force use of /data for persistence regardless of other settings. - if Path("/data").exists(): - self.index_dir = Path("/data/index") - self.sessions_dir = Path("/data/sessions") - return self - - index_dir: Path = Field(default=Path("data/index"), alias="INDEX_DIR") - sessions_dir: Path = Field(default=Path("data/sessions"), alias="SESSIONS_DIR") - chunk_size_tokens: int = 350 # Reduced for TPM safety - chunk_overlap_tokens: int = 80 - top_k: int = Field(default=20, alias="TOP_K") - max_context_chunks: int = 10 # Increased to 10 to ensure all list items (like leaves) are captured - request_timeout_s: float = 20.0 - cors_allow_origins: str = Field(default="*", alias="CORS_ALLOW_ORIGINS") - api_key: str = Field(default="", alias="API_KEY") - rate_limit_requests: int = Field(default=60, alias="RATE_LIMIT_REQUESTS") - rate_limit_window_seconds: int = Field(default=60, alias="RATE_LIMIT_WINDOW_SECONDS") - - # SMTP Settings for OTP - smtp_server: str = Field(default="smtp.gmail.com", alias="SMTP_SERVER") - smtp_port: int = Field(default=465, alias="SMTP_PORT") - smtp_user: str = Field(default="", alias="SMTP_USER") - smtp_pass: str = Field(default="", alias="SMTP_PASS") - admin_email: str = Field(default="randomjoedown@gmail.com", alias="ADMIN_EMAIL") - - -@lru_cache -def get_settings() -> Settings: - return Settings() diff --git a/backup_2026-05-04/app/main.py b/backup_2026-05-04/app/main.py deleted file mode 100644 index 72a9ae8d86ae1def15905d831129fe0ba5b98a72..0000000000000000000000000000000000000000 --- a/backup_2026-05-04/app/main.py +++ /dev/null @@ -1,196 +0,0 @@ -import logging - -from fastapi import FastAPI, HTTPException, Header, Request -from fastapi.responses import RedirectResponse -from fastapi.middleware.cors import CORSMiddleware -from fastapi.staticfiles import StaticFiles -import os - -from app.api.schemas import ChatRequest, ChatResponse, HealthResponse, SessionUpdateNameRequest -from app.core.config import get_settings -from app.services.embeddings import EmbeddingService -from app.services.llm import LLMService -from app.services.rag_pipeline import RAGPipeline -from app.services.rate_limiter import InMemoryRateLimiter -from app.services.vector_store import FaissVectorStore -from app.services.reranker import RerankerService -from app.ui_gradio import demo -from app.admin.router import admin_router -from app.services import session_store as _ss -import asyncio -import gradio as gr - -logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(name)s - %(message)s") -logger = logging.getLogger(__name__) - -settings = get_settings() -embedding_service = EmbeddingService(settings.embedding_model) -vector_store = FaissVectorStore( - embedding_service=embedding_service, - docs_dir=settings.docs_dir, - index_dir=settings.index_dir, - chunk_size_tokens=settings.chunk_size_tokens, - chunk_overlap_tokens=settings.chunk_overlap_tokens, -) -llm_service = LLMService( - provider=settings.llm_provider, - groq_api_key=settings.groq_api_key, - groq_model=settings.groq_model, - groq_rewrite_model=settings.groq_rewrite_model, - hf_api_key=settings.hf_api_key, - hf_model=settings.hf_model, - timeout_s=settings.request_timeout_s, -) -reranker_service = RerankerService(settings.reranker_model) -pipeline = RAGPipeline( - vector_store=vector_store, - llm_service=llm_service, - reranker=reranker_service, - top_k=settings.top_k, - max_context_chunks=settings.max_context_chunks -) - -# ── FastAPI App ─────────────────────────────────────────────────────────────── -fastapi_app = FastAPI(title=settings.app_name) - -allow_origins = [o.strip() for o in settings.cors_allow_origins.split(",") if o.strip()] -fastapi_app.add_middleware( - CORSMiddleware, - allow_origins=["*"], # Explicitly allow all for testing - allow_credentials=True, - allow_methods=["*"], - allow_headers=["*"], -) - -rate_limiter = InMemoryRateLimiter(settings.rate_limit_requests, settings.rate_limit_window_seconds) - -# ── Admin Panel (mounted on FastAPI before Gradio wrapping) ─────────────────── -fastapi_app.include_router(admin_router) - -# ── Static Files ────────────────────────────────────────────────────────────── -static_dir = os.path.join(os.path.dirname(os.path.dirname(__file__)), "static") -if os.path.exists(static_dir): - fastapi_app.mount("/static", StaticFiles(directory=static_dir), name="static") - -@fastapi_app.get("/widget", include_in_schema=False) -async def get_widget(): - """Serves the chat widget HTML with explicit CORS headers.""" - from fastapi.responses import FileResponse - path = os.path.join(static_dir, "addon.html") - if not os.path.exists(path): - raise HTTPException(status_code=404, detail="Widget not found") - return FileResponse( - path, - headers={ - "Access-Control-Allow-Origin": "*", - "Cache-Control": "no-cache, no-store, must-revalidate" - } - ) - - -@fastapi_app.on_event("startup") -def startup_event() -> None: - logger.info("─── RAG SERVICE STARTUP ───") - logger.info("Index Directory: %s", settings.index_dir) - logger.info("Sessions Directory: %s", settings.sessions_dir) - - # Verify persistence mount - if str(settings.index_dir).startswith("/data"): - if os.path.exists("/data"): - logger.info("✅ Persistent storage /data detected and in use.") - else: - logger.warning("⚠️ /data requested but NOT found on filesystem!") - else: - logger.warning("⚠️ Using ephemeral storage (data will be lost on rebuild).") - - logger.info("Loading/building FAISS index...") - vector_store.build_or_load() - logger.info("RAG service started with %s chunks", len(vector_store.metadata)) - - -@fastapi_app.get("/", include_in_schema=False) -async def root(): - """Prevent direct browser access to the root URL.""" - raise HTTPException( - status_code=403, - detail="Direct access to this resource is restricted. Please use the authorized interface." - ) - - -@fastapi_app.get("/health", response_model=HealthResponse) -def health() -> HealthResponse: - h = vector_store.health() - return HealthResponse(status="ok", **h) - - -@fastapi_app.post("/api/chat", response_model=ChatResponse) -async def chat( - payload: ChatRequest, - request: Request, - x_api_key: str | None = Header(default=None), -) -> ChatResponse: - if not payload.message.strip(): - raise HTTPException(status_code=400, detail="message must not be empty") - if settings.api_key and x_api_key != settings.api_key: - raise HTTPException(status_code=401, detail="unauthorized") - - client_ip = request.client.host if request.client else "unknown" - if not rate_limiter.allow(client_ip): - raise HTTPException(status_code=429, detail="rate limit exceeded") - - logger.info("Incoming chat request: msg='%s', sid='%s'", payload.message[:30], payload.session_id) - - # Fetch user_name if available for personalization - user_name = None - if payload.session_id: - sess_data = await _ss.get_session(payload.session_id) - user_name = sess_data.get("user_name") - - try: - result = await pipeline.chat(payload.message, payload.history, user_name=user_name) - - # ── Fire-and-forget: persist to session store in background ── - if payload.session_id: - asyncio.create_task(_ss.save_message(payload.session_id, payload.message, result["reply"])) - - return ChatResponse(**result) - except Exception as e: - import httpx - import traceback - logger.error(f"Error in chat endpoint: {str(e)}\n{traceback.format_exc()}") - - error_msg = "⚠️ Oops! Something went wrong." - if isinstance(e, httpx.HTTPStatusError): - if e.response.status_code == 429: - error_msg = "⚠️ Rate limit reached. Please slow down a bit!" - else: - error_msg = "⚠️ I encountered an error. Please try again in a few seconds." - - return ChatResponse(reply=error_msg, retrieved_chunks=[]) - - -@fastapi_app.post("/api/session/update-name") -async def update_session_user_name(payload: SessionUpdateNameRequest): - """Updates the user name and renames the session file to match the name.""" - new_id = await _ss.rename_session(payload.session_id, payload.user_name) - return {"status": "ok", "new_session_id": new_id} - - -@fastapi_app.get("/api/session/history/{session_id}") -async def get_session_history(session_id: str): - """Returns the chat history for a session formatted for the frontend.""" - data = await _ss.get_session(session_id) - if not data: - return {"history": []} - - history = [] - for msg in data.get("messages", []): - history.append({"role": "user", "content": msg["question"]}) - history.append({"role": "assistant", "content": msg["answer"]}) - return {"history": history, "user_name": data.get("user_name", "")} - - -# ── Mount Gradio at /ui — MUST be last, wraps the FastAPI app ───────────────── -# All FastAPI routes (including /admin) are already registered above and are -# preserved inside the Gradio-wrapped Starlette app. -app = gr.mount_gradio_app(fastapi_app, demo, path="/ui") diff --git a/backup_2026-05-04/app/services/__init__.py b/backup_2026-05-04/app/services/__init__.py deleted file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..0000000000000000000000000000000000000000 diff --git a/backup_2026-05-04/app/services/chunker.py b/backup_2026-05-04/app/services/chunker.py deleted file mode 100644 index d033e1f5f243452dae87fb2a556d8d9e593b854f..0000000000000000000000000000000000000000 --- a/backup_2026-05-04/app/services/chunker.py +++ /dev/null @@ -1,39 +0,0 @@ -from typing import List, Dict - - -def _tokenize(text: str) -> List[str]: - return text.split() - - -def chunk_documents( - documents: List[Dict[str, str]], - chunk_size_tokens: int = 420, - chunk_overlap_tokens: int = 80, -) -> List[Dict[str, str]]: - chunks: List[Dict[str, str]] = [] - step = max(1, chunk_size_tokens - chunk_overlap_tokens) - - for doc in documents: - tokens = _tokenize(doc["text"]) - if not tokens: - continue - - i = 0 - chunk_id = 0 - while i < len(tokens): - window = tokens[i : i + chunk_size_tokens] - if not window: - break - - chunk_text = " ".join(window) - chunks.append( - { - "id": f"{doc['source']}::chunk_{chunk_id}", - "source": doc["source"], - "text": chunk_text, - } - ) - chunk_id += 1 - i += step - - return chunks diff --git a/backup_2026-05-04/app/services/document_loader.py b/backup_2026-05-04/app/services/document_loader.py deleted file mode 100644 index cef3783744611ed6c925a98f48033872eb704d91..0000000000000000000000000000000000000000 --- a/backup_2026-05-04/app/services/document_loader.py +++ /dev/null @@ -1,37 +0,0 @@ -from pathlib import Path -from typing import List - -from pypdf import PdfReader - - -def _clean_text(text: str) -> str: - lines = [line.strip() for line in text.splitlines() if line.strip()] - merged = " ".join(lines) - return " ".join(merged.split()) - - -def load_documents(docs_dir: Path) -> List[dict]: - docs: List[dict] = [] - if not docs_dir.exists(): - return docs - - for file_path in sorted(docs_dir.rglob("*")): - if not file_path.is_file(): - continue - - suffix = file_path.suffix.lower() - if suffix == ".txt": - raw = file_path.read_text(encoding="utf-8", errors="ignore") - cleaned = _clean_text(raw) - if cleaned: - docs.append({"source": str(file_path), "text": cleaned}) - elif suffix == ".pdf": - reader = PdfReader(str(file_path)) - pages = [] - for page in reader.pages: - pages.append(page.extract_text() or "") - cleaned = _clean_text("\n".join(pages)) - if cleaned: - docs.append({"source": str(file_path), "text": cleaned}) - - return docs diff --git a/backup_2026-05-04/app/services/embeddings.py b/backup_2026-05-04/app/services/embeddings.py deleted file mode 100644 index a920c0ffb22715a7eeefe8790c45ebb7b67cdaaf..0000000000000000000000000000000000000000 --- a/backup_2026-05-04/app/services/embeddings.py +++ /dev/null @@ -1,34 +0,0 @@ -from typing import List -from functools import lru_cache - -import numpy as np -from sentence_transformers import SentenceTransformer - - -class EmbeddingService: - def __init__(self, model_name: str) -> None: - self.model = SentenceTransformer(model_name) - - def encode(self, texts: List[str]) -> np.ndarray: - vectors = self.model.encode( - texts, - normalize_embeddings=True, - show_progress_bar=False, - convert_to_numpy=True, - ) - return np.asarray(vectors, dtype=np.float32) - - @lru_cache(maxsize=2048) - def encode_query_cached(self, text: str) -> bytes: - vec = self.model.encode( - [text], - normalize_embeddings=True, - show_progress_bar=False, - convert_to_numpy=True, - ) - arr = np.asarray(vec, dtype=np.float32) - return arr.tobytes() - - def encode_query(self, text: str) -> np.ndarray: - buf = self.encode_query_cached(text) - return np.frombuffer(buf, dtype=np.float32).reshape(1, -1) diff --git a/backup_2026-05-04/app/services/llm.py b/backup_2026-05-04/app/services/llm.py deleted file mode 100644 index 41ee086af591bfd93cf32ca8a3b74431bbbafca7..0000000000000000000000000000000000000000 --- a/backup_2026-05-04/app/services/llm.py +++ /dev/null @@ -1,264 +0,0 @@ -from typing import List, Dict -import httpx -import re - -# ═══════════════════════════════════════════════════════════════════════ -# MASTER SYSTEM PROMPT — Martechsol HR Assistant -# Intelligence: Understand Intent → Retrieve Facts → Respond Precisely -# ═══════════════════════════════════════════════════════════════════════ -SYSTEM_PROMPT = """You are the Martechsol HR Assistant — intelligent, precise, and formal. - -━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ -STEP 1 — UNDERSTAND THE INTENT -━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ -Read the question carefully. Identify the SINGLE core topic being asked. Apply intelligent defaults: -• "timing" / "timings" (no context) → office working hours ONLY — not payment or any other timing -• "leaves" / "leave" (no context) → leave names + day counts ONLY — NOT leave policies or eligibility -• "paid leaves" / "all leaves" → enumerate EVERY leave type with its name and count -• "salary" / "pay" (no context) → salary structure or amount — NOT payment date unless explicitly asked -• "benefits" / "perks" / "allowances" → list EVERY benefit with its name and value -• "terminate" / "termination" → resignation/termination procedure — NOT general policies -If a question has an obvious workplace context, always default to the most common interpretation. - -━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ -STEP 2 — STRICT SCOPE DISCIPLINE -━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ -Answer ONLY what was asked. NEVER expand into: -• Policies, approval processes, eligibility rules, or consequences — unless user asks for policy/process -• Related topics the user did not mention -• Broad overviews when a specific fact was requested -• Context that wasn't in the question - -Scope examples: - "How many sick leaves?" → ONE number. Not the sick leave policy. - "What are office timings?" → ONE sentence with time range. Not break times or exceptions. - "What are paid leaves?" → Complete list of all paid leave types + counts. Not rules for taking them. - "Tell me about maternity leave" → Count + brief key fact. Not full policies unless asked. - -━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ -STEP 3 — CHOOSE THE RIGHT FORMAT -━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ - -FORMAT A — SINGLE FACT (default for most questions) - When: One direct answer needed (timings, a specific leave count, a single number/date) - Rule: ONE complete sentence. Maximum 25–30 words. Never cut mid-sentence. - Example: Office hours are 9:00 AM to 6:00 PM, Monday to Saturday. - -FORMAT B — EXHAUSTIVE LIST - When: User asks for ALL items in a category ("all leaves", "all benefits", "list all X", "paid leaves") - Rule: - • Include EVERY single item found — omitting even one is FORBIDDEN - • One item per line: Item Name: value
- • No intro sentence, no closing sentence, no extra commentary - • Continue until ALL items are listed - Example: - Casual Leave: 10 days
- Sick Leave: 10 days
- Annual Leave: 14 days
- Maternity Leave: 90 days
- Paternity Leave: 3 days
- Hajj Leave: 30 days
- ...(list every item — do NOT stop early) - -FORMAT C — BRIEF EXPLANATION - When: User asks HOW something works, or asks for a process/procedure - Rule: Maximum 3 bullet points. Each bullet = one complete, factual sentence. No filler words. - -━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ -STRICT QUALITY RULES -━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ -✓ ZERO hallucination — every fact must exist in Expert Data only. No guessing. -✓ If not in Expert Data → reply exactly: "I don't have that information." -✓ Never cut a sentence mid-way — always complete every sentence fully -✓ NEVER mention: "document", "handbook", "manual", "policy file", or any source reference -✓ Use bold for names, numbers, dates, leave types, and all key terms -✓ Use
between list items for clean vertical spacing -✓ Tone: formal, warm, and professional — never robotic, never chatty -✓ Do NOT add greetings, closings, or "Is there anything else?" type phrases""" - - -def _build_context(chunks: List[Dict[str, str]], max_words: int = 1500) -> str: - """Combines retrieved chunks into a clean context string, capped at max_words. - Prevents TPM spikes when chunks are unexpectedly large.""" - if not chunks: - return "" - parts = [] - total_words = 0 - for c in chunks: - text = c['text'] - word_count = len(text.split()) - if total_words + word_count > max_words: - # Add a trimmed version of this chunk if possible - remaining = max_words - total_words - if remaining > 30: # Only worth adding if meaningful content remains - trimmed_words = text.split()[:remaining] - parts.append(" ".join(trimmed_words)) - break - parts.append(text) - total_words += word_count - return "\n\n".join(parts) - - -class LLMService: - def __init__( - self, - provider: str, - groq_api_key: str, - groq_model: str, - groq_rewrite_model: str, - hf_api_key: str, - hf_model: str, - timeout_s: float = 20.0, - ) -> None: - self.provider = provider.lower().strip() - self.groq_api_key = groq_api_key - self.groq_model = groq_model - self.groq_rewrite_model = groq_rewrite_model - self.hf_api_key = hf_api_key - self.hf_model = hf_model - self.timeout_s = timeout_s - - async def generate_multi_queries(self, query: str, history: List[Dict[str, str]]) -> List[str]: - """Generates multiple search queries to capture broader context from the document.""" - prompt = f"""You are an intelligent HR search optimizer. A user asked: "{query}" - -STEP 1 — UNDERSTAND THE INTENT: -What is the user ACTUALLY asking about? Apply workplace common sense: -- "timing" / "timings" (no qualifier) = office working hours, workday schedule -- "leaves" / "paid leaves" / "all leaves" = all leave types available to employees -- "salary" / "pay" = salary structure or amount -- "benefits" / "allowances" = employee benefits and perks -- "terminate" / "resign" = termination or resignation process - -STEP 2 — GENERATE 3 TARGETED SEARCH QUERIES: -Write 3 diverse search queries to retrieve ALL relevant information from an employee handbook. -For the identified intent, cover synonyms, related terms, and sub-categories: -- For office timings → search: working hours, office schedule, workday timing, shift hours -- For leaves → Query 1 MUST be a keyword dump: "casual leave sick leave annual leave maternity paternity hajj bereavement study unauthorized" -- For salary → cover: salary structure, payroll, compensation, increments, deductions -- For benefits → cover: allowances, perks, medical, bonuses, reimbursements - -Output ONLY 3 queries, one per line. No numbers, no bullets, no explanations. - -Queries:""" - - try: - resp = await self._call_groq( - prompt, [], - "You output only search queries, one per line. No explanations, no numbering.", - model_override=self.groq_rewrite_model - ) - queries = [ - q.strip().lstrip('0123456789.-) ').strip() - for q in resp.split("\n") - if q.strip() and len(q.strip()) > 3 - ] - # Always include original query - if query not in queries: - queries.append(query) - return queries[:4] - except Exception: - return [query] - - async def answer( - self, - question: str, - chunks: List[Dict[str, str]], - history: List[Dict[str, str]], - user_name: str = None - ) -> str: - # Keep only last 4 messages (2 turns) for TPM safety - pruned_history = history[-4:] - context = _build_context(chunks) - - # Build system message - system_msg = SYSTEM_PROMPT - if user_name and user_name not in ("default name", "", None): - system_msg += f"\n\nYou are speaking with {user_name}." - - # Build user prompt — clean and direct - if not context: - user_prompt = question - else: - user_prompt = f"Expert Data:\n{context}\n\nQuestion: {question}" - - return await self._call_groq(user_prompt, pruned_history, system_msg) - - async def _call_groq( - self, - user_prompt: str, - history: List[Dict[str, str]], - system_msg: str, - model_override: str = None - ) -> str: - if not self.groq_api_key: - return "Expert access required." - - url = "https://api.groq.com/openai/v1/chat/completions" - headers = { - "Authorization": f"Bearer {self.groq_api_key}", - "Content-Type": "application/json" - } - - messages = [{"role": "system", "content": system_msg}] - for msg in history: - messages.append({"role": msg["role"], "content": msg["content"]}) - messages.append({"role": "user", "content": user_prompt}) - - target_model = model_override or self.groq_model - - # Qwen3 and DeepSeek-R1 are thinking models - is_thinking_model = any(k in target_model.lower() for k in ["deepseek", "qwen3", "qwen/qwen3"]) - temp = 0.6 if is_thinking_model else 0.0 - - # For Qwen3: suppress thinking mode on answer calls to reduce latency & TPM usage. - # /no_think is Qwen3's built-in signal to skip chain-of-thought reasoning. - # We ONLY suppress for the main answer model, not the rewrite model. - if is_thinking_model and model_override is None: - messages[-1]["content"] = "/no_think\n\n" + messages[-1]["content"] - - payload = { - "model": target_model, - "temperature": temp, - "max_tokens": 512, # Enough for complete listings with HTML, well within TPM limits - "messages": messages, - } - - async with httpx.AsyncClient(timeout=self.timeout_s) as client: - resp = await client.post(url, headers=headers, json=payload) - if resp.status_code >= 400: - import logging - logging.getLogger(__name__).error( - f"Groq API Error {resp.status_code}: {resp.text}" - ) - resp.raise_for_status() - data = resp.json() - content = data["choices"][0]["message"]["content"].strip() - - # ── Post-Processing: Strip all internal reasoning artifacts ── - - # 1. Strip ... blocks (Qwen3, DeepSeek-R1) - content = re.sub(r'.*?', '', content, flags=re.DOTALL).strip() - - # 2. Strip leading conversational filler (single line only, not entire content) - content = re.sub( - r'^(Okay[,.]?\s*|Alright[,.]?\s*|Sure[,.]?\s*|Let\'s see[,.]?\s*|' - r'Based on (?:the )?(?:provided |available )?(?:data|information|context)[,.]?\s*|' - r'According to (?:the )?(?:provided |available )?(?:data|information)[,.]?\s*)', - '', content, flags=re.IGNORECASE - ).strip() - - # 3. Remove lines that are pure internal self-talk (only if they appear alone at start) - lines = content.split('\n') - filtered = [] - for i, line in enumerate(lines): - is_self_talk = bool(re.match( - r'^\s*(I need to|I will|I should|I\'m going to|Let me|Now I|First,? I|' - r'I\'ll|The user is asking|The question is about)', - line, re.IGNORECASE - )) - if not is_self_talk: - filtered.append(line) - content = '\n'.join(filtered).strip() - - return content diff --git a/backup_2026-05-04/app/services/rag_pipeline.py b/backup_2026-05-04/app/services/rag_pipeline.py deleted file mode 100644 index cdd48d9a800a6a4291bb2b739d5e43dc55206b6a..0000000000000000000000000000000000000000 --- a/backup_2026-05-04/app/services/rag_pipeline.py +++ /dev/null @@ -1,98 +0,0 @@ -import logging -from typing import Dict, List - -from app.services.llm import LLMService -from app.services.vector_store import FaissVectorStore -from app.services.reranker import RerankerService - -logger = logging.getLogger(__name__) - -# RRF scores are small (e.g. 0.016–0.033), so threshold must be very low -RELEVANCE_THRESHOLD = 0.01 -# Cross-encoder logit > 0 means > 50% relevance probability -RERANK_THRESHOLD = 0.0 -# If ALL chunks fail rerank threshold, fall back to this many top chunks -RERANK_FALLBACK_N = 2 - - -class RAGPipeline: - def __init__( - self, - vector_store: FaissVectorStore, - llm_service: LLMService, - reranker: RerankerService, - top_k: int, - max_context_chunks: int - ) -> None: - self.vector_store = vector_store - self.llm_service = llm_service - self.reranker = reranker - self.top_k = top_k - self.max_context_chunks = max_context_chunks - - async def chat(self, message: str, history: List[Dict[str, str]], user_name: str = None) -> Dict[str, object]: - # ── Step 1: Generate multiple search queries for broad coverage ── - queries = await self.llm_service.generate_multi_queries(message, history) - logger.info("Generated %d queries for: '%s' → %s", len(queries), message[:40], queries) - - # ── Step 2: Collect unique chunks across all queries ── - seen_ids = set() - all_retrieved = [] - - for q in queries: - query_chunks = self.vector_store.search(q, top_k=self.top_k) - for chunk in query_chunks: - if chunk["id"] not in seen_ids: - all_retrieved.append(chunk) - seen_ids.add(chunk["id"]) - - # ── Step 3: Initial relevance filter ── - initial_chunks = [c for c in all_retrieved if c["score"] >= RELEVANCE_THRESHOLD] - - if not initial_chunks: - logger.info("No relevant chunks found for: '%s' — returning no-info response", message) - # Pass empty chunks; LLM is instructed to say "I don't have that information" - reply = await self.llm_service.answer( - question=message, - chunks=[], - history=history, - user_name=user_name - ) - return {"reply": reply, "retrieved_chunks": []} - - # ── Step 4: Deep reranking via Cross-Encoder ── - # Enrihc the reranker query with the LLM's expanded search terms - rerank_query = message - if len(queries) > 0 and queries[0] != message: - rerank_query = f"{message} {queries[0]}" - - reranked_chunks = self.reranker.rerank(rerank_query, initial_chunks, top_n=self.max_context_chunks) - - # Filter by rerank score threshold - final_chunks = [c for c in reranked_chunks if c.get("rerank_score", 0) > RERANK_THRESHOLD] - - # ── Smart Fallback: if ALL chunks fail the threshold, use the top N anyway ── - # This prevents the bot from saying "I don't have info" when content WAS retrieved - # but the cross-encoder wasn't confident enough (e.g. paraphrased queries) - if not final_chunks and reranked_chunks: - final_chunks = reranked_chunks[:RERANK_FALLBACK_N] - logger.info( - "Rerank fallback activated — all chunks below threshold (best score=%.3f), using top %d", - reranked_chunks[0].get("rerank_score", 0), - len(final_chunks) - ) - - logger.info( - "Pipeline: retrieved=%d → relevance_filtered=%d → reranked=%d → final=%d (best_score=%.3f)", - len(all_retrieved), len(initial_chunks), len(reranked_chunks), len(final_chunks), - reranked_chunks[0].get("rerank_score", 0) if reranked_chunks else 0.0 - ) - - # ── Step 5: Generate answer with top-ranked context ── - reply = await self.llm_service.answer( - question=message, - chunks=final_chunks, - history=history, - user_name=user_name - ) - return {"reply": reply, "retrieved_chunks": final_chunks} diff --git a/backup_2026-05-04/app/services/rate_limiter.py b/backup_2026-05-04/app/services/rate_limiter.py deleted file mode 100644 index 013800657d9cdbd132919274e6fae6076345bd5a..0000000000000000000000000000000000000000 --- a/backup_2026-05-04/app/services/rate_limiter.py +++ /dev/null @@ -1,24 +0,0 @@ -import time -from collections import defaultdict, deque -from typing import Deque, Dict - - -class InMemoryRateLimiter: - def __init__(self, max_requests: int, window_seconds: int) -> None: - self.max_requests = max_requests - self.window_seconds = window_seconds - self._hits: Dict[str, Deque[float]] = defaultdict(deque) - - def allow(self, key: str) -> bool: - now = time.time() - window_start = now - self.window_seconds - q = self._hits[key] - - while q and q[0] < window_start: - q.popleft() - - if len(q) >= self.max_requests: - return False - - q.append(now) - return True diff --git a/backup_2026-05-04/app/services/reranker.py b/backup_2026-05-04/app/services/reranker.py deleted file mode 100644 index c7355c45b8fbe66cd53f8303744fdc8ffbac7969..0000000000000000000000000000000000000000 --- a/backup_2026-05-04/app/services/reranker.py +++ /dev/null @@ -1,26 +0,0 @@ -from typing import List, Dict -import numpy as np -from sentence_transformers import CrossEncoder - -class RerankerService: - def __init__(self, model_name: str) -> None: - self.model = CrossEncoder(model_name) - - def rerank(self, query: str, chunks: List[Dict[str, str]], top_n: int = 4) -> List[Dict[str, str]]: - if not chunks: - return [] - - # Prepare pairs for cross-encoder - pairs = [[query, chunk["text"]] for chunk in chunks] - - # Get scores - scores = self.model.predict(pairs) - - # Add scores to chunks and sort - for i, chunk in enumerate(chunks): - chunk["rerank_score"] = float(scores[i]) - - # Sort by rerank_score descending - sorted_chunks = sorted(chunks, key=lambda x: x["rerank_score"], reverse=True) - - return sorted_chunks[:top_n] diff --git a/backup_2026-05-04/app/services/session_store.py b/backup_2026-05-04/app/services/session_store.py deleted file mode 100644 index 0f9e07a28f9e0dcb6e6f11c282d79dd203eb4bab..0000000000000000000000000000000000000000 --- a/backup_2026-05-04/app/services/session_store.py +++ /dev/null @@ -1,284 +0,0 @@ -""" -session_store.py -──────────────── -Async, fire-and-forget JSON-based chat session storage. -Each browser session gets its own JSON file in data/sessions/. - -Rules: - • NEVER raises exceptions that propagate upward — any failure is silently logged. - • ALL disk I/O is done with asyncio to avoid blocking the event loop. - • Thread-safe: uses an asyncio.Lock per session_id to avoid race conditions. -""" - -import asyncio -import json -import logging -import os -from datetime import datetime, timezone -from pathlib import Path -from typing import Dict - -from app.core.config import get_settings -logger = logging.getLogger(__name__) -settings = get_settings() - -# ── Directory where session JSON files are stored ─────────────────────────── -def _get_sessions_dir() -> Path: - """Returns the current sessions directory from settings.""" - return settings.sessions_dir - - -# ── Per-session locks to prevent concurrent write corruption ───────────────── -_session_locks: Dict[str, asyncio.Lock] = {} -_locks_meta_lock = asyncio.Lock() # protects the _session_locks dict itself - - -def _get_session_path(session_id: str) -> Path: - """Returns the absolute path for a session's JSON file.""" - return _get_sessions_dir() / f"{session_id}.json" - - -def _now_iso() -> str: - """Returns the current UTC time as an ISO-8601 string.""" - return datetime.now(timezone.utc).isoformat() - - -async def _get_lock(session_id: str) -> asyncio.Lock: - """Gets or creates a per-session asyncio.Lock in a thread-safe manner.""" - async with _locks_meta_lock: - if session_id not in _session_locks: - _session_locks[session_id] = asyncio.Lock() - return _session_locks[session_id] - - -def _ensure_sessions_dir() -> None: - """Creates the sessions directory if it does not exist. Sync, called once.""" - try: - _get_sessions_dir().mkdir(parents=True, exist_ok=True) - except OSError as exc: - logger.error("session_store: cannot create sessions dir: %s", exc) - - -# Call once at import time so the directory is always ready. -_ensure_sessions_dir() - - -async def ensure_session(session_id: str) -> None: - """ - Creates the session JSON file if it doesn't exist yet. - Safe to call on every page load — does nothing if file exists. - """ - path = _get_session_path(session_id) - if path.exists(): - return - lock = await _get_lock(session_id) - async with lock: - # Double-check after acquiring lock - if path.exists(): - return - skeleton = { - "session_id": session_id, - "user_name": "default name", - "created_at": _now_iso(), - "last_active": _now_iso(), - "message_count": 0, - "messages": [], - } - try: - await asyncio.to_thread(_write_json, path, skeleton) - logger.info("session_store: created session %s", session_id) - except Exception as exc: - logger.error("session_store: failed to create session %s: %s", session_id, exc) - - -async def save_message(session_id: str, question: str, answer: str) -> None: - """ - Appends a Question+Answer pair with timestamps to the session's JSON file. - Fire-and-forget safe — caller should wrap in asyncio.create_task(). - """ - path = _get_session_path(session_id) - logger.info("session_store: save_message called for session_id=%s, path=%s", session_id, path) - lock = await _get_lock(session_id) - async with lock: - try: - # Load current data - if path.exists(): - data = await asyncio.to_thread(_read_json, path) - else: - # Session file missing — recreate it gracefully - data = { - "session_id": session_id, - "user_name": "default name", - "created_at": _now_iso(), - "last_active": _now_iso(), - "message_count": 0, - "messages": [], - } - - timestamp = _now_iso() - data["messages"].append({ - "id": data["message_count"] + 1, - "timestamp": timestamp, - "question": question, - "answer": answer, - }) - data["message_count"] += 1 - data["last_active"] = timestamp - - await asyncio.to_thread(_write_json, path, data) - logger.info("session_store: successfully saved message to %s", path) - except Exception as exc: - logger.error( - "session_store: failed to save message for session %s: %s", - session_id, - exc, - ) - - -async def get_all_sessions() -> list: - """ - Returns a list of all session summaries (metadata only, no full messages). - Used by the Admin Panel sidebar. - """ - try: - files = await asyncio.to_thread(_list_json_files) - sessions = [] - for fpath in files: - try: - data = await asyncio.to_thread(_read_json, fpath) - sessions.append({ - "session_id": data.get("session_id", fpath.stem), - "user_name": data.get("user_name", ""), - "created_at": data.get("created_at", ""), - "last_active": data.get("last_active", ""), - "message_count": data.get("message_count", 0), - }) - except Exception as exc: - logger.warning("session_store: could not read %s: %s", fpath, exc) - # Sort newest first - sessions.sort(key=lambda s: s.get("last_active", ""), reverse=True) - return sessions - except Exception as exc: - logger.error("session_store: get_all_sessions failed: %s", exc) - return [] - - -async def get_session(session_id: str) -> dict: - """ - Returns the full session data (including all messages) for the Admin Panel. - Returns an empty dict if the session does not exist. - """ - path = _get_session_path(session_id) - try: - if not path.exists(): - return {} - return await asyncio.to_thread(_read_json, path) - except Exception as exc: - logger.error("session_store: get_session(%s) failed: %s", session_id, exc) - return {} - - -async def delete_session(session_id: str) -> bool: - """ - Deletes a session file. Returns True on success, False on failure. - Used by Admin Panel if needed in the future. - """ - path = _get_session_path(session_id) - lock = await _get_lock(session_id) - async with lock: - try: - if path.exists(): - await asyncio.to_thread(os.remove, path) - return True - except Exception as exc: - logger.error("session_store: delete_session(%s) failed: %s", session_id, exc) - return False - - -async def update_session_name(session_id: str, user_name: str) -> None: - """Updates the user_name field in the session JSON file.""" - path = _get_session_path(session_id) - lock = await _get_lock(session_id) - async with lock: - try: - if path.exists(): - data = await asyncio.to_thread(_read_json, path) - data["user_name"] = user_name - await asyncio.to_thread(_write_json, path, data) - logger.info("session_store: updated name for session %s to %s", session_id, user_name) - else: - # If session doesn't exist yet, we can't update it, but we could create it? - # Usually ensure_session is called first. - logger.warning("session_store: attempt to update name for non-existent session %s", session_id) - except Exception as exc: - logger.error("session_store: failed to update name for session %s: %s", session_id, exc) - - -async def rename_session(old_id: str, new_name: str) -> str: - """ - Renames a session file from old_id to a name based on new_name. - Returns the new session_id used. - """ - import re - # 1. Sanitize the name for use as a filename - clean_name = re.sub(r'[^\w\s-]', '', new_name).strip().replace(' ', '_') - if not clean_name: - clean_name = "User" - - new_id = clean_name - old_path = _get_session_path(old_id) - new_path = _get_session_path(new_id) - - # Ensure uniqueness: if John_Doe exists, try John_Doe_1, etc. - counter = 1 - while new_path.exists() and new_id != old_id: - new_id = f"{clean_name}_{counter}" - new_path = _get_session_path(new_id) - counter += 1 - - if new_id == old_id: - return old_id - - # Lock both IDs to prevent race conditions during move - lock_old = await _get_lock(old_id) - lock_new = await _get_lock(new_id) - - async with lock_old: - async with lock_new: - try: - if old_path.exists(): - data = await asyncio.to_thread(_read_json, old_path) - data["session_id"] = new_id - data["user_name"] = new_name - - # Write to new path and delete old - await asyncio.to_thread(_write_json, new_path, data) - await asyncio.to_thread(os.remove, old_path) - - logger.info("session_store: renamed session %s -> %s", old_id, new_id) - return new_id - else: - logger.warning("session_store: cannot rename non-existent session %s", old_id) - return old_id - except Exception as exc: - logger.error("session_store: failed to rename session %s: %s", old_id, exc) - return old_id - - -# ── Sync helper functions (run via asyncio.to_thread) ──────────────────────── - -def _write_json(path: Path, data: dict) -> None: - """Atomically writes JSON to a file using a temp file + rename pattern.""" - tmp_path = path.with_suffix(".tmp") - with open(tmp_path, "w", encoding="utf-8") as f: - json.dump(data, f, ensure_ascii=False, indent=2) - os.replace(tmp_path, path) # Atomic on POSIX, best-effort on Windows - - -def _read_json(path: Path) -> dict: - with open(path, "r", encoding="utf-8") as f: - return json.load(f) - - -def _list_json_files() -> list: - return list(_get_sessions_dir().glob("*.json")) diff --git a/backup_2026-05-04/app/services/vector_store.py b/backup_2026-05-04/app/services/vector_store.py deleted file mode 100644 index e31029806d59d189af5f57cc6bb71d4759a74712..0000000000000000000000000000000000000000 --- a/backup_2026-05-04/app/services/vector_store.py +++ /dev/null @@ -1,167 +0,0 @@ -import hashlib -import json -from pathlib import Path -from typing import Dict, List, Tuple - -import faiss -import numpy as np -import pickle -from rank_bm25 import BM25Okapi - -from app.services.chunker import chunk_documents -from app.services.document_loader import load_documents -from app.services.embeddings import EmbeddingService - - -class FaissVectorStore: - def __init__( - self, - embedding_service: EmbeddingService, - docs_dir: Path, - index_dir: Path, - chunk_size_tokens: int, - chunk_overlap_tokens: int, - ) -> None: - self.embedding_service = embedding_service - self.docs_dir = docs_dir - self.index_dir = index_dir - self.chunk_size_tokens = chunk_size_tokens - self.chunk_overlap_tokens = chunk_overlap_tokens - - self.faiss_index_path = index_dir / "faiss.index" - self.bm25_index_path = index_dir / "bm25.pkl" - self.metadata_path = index_dir / "metadata.json" - self.state_path = index_dir / "state.json" - - self.index = None - self.bm25 = None - self.metadata: List[Dict[str, str]] = [] - self.docs_loaded = False - self.last_retrieved: List[Dict[str, str]] = [] - - def _compute_docs_fingerprint(self) -> str: - hasher = hashlib.sha256() - # Include chunk settings in fingerprint so changing them triggers re-index - hasher.update(str(self.chunk_size_tokens).encode("utf-8")) - hasher.update(str(self.chunk_overlap_tokens).encode("utf-8")) - - if not self.docs_dir.exists(): - return "no_docs" - for path in sorted(self.docs_dir.rglob("*")): - if path.is_file() and path.suffix.lower() in {".txt", ".pdf"}: - stat = path.stat() - hasher.update(str(path).encode("utf-8")) - hasher.update(str(stat.st_mtime_ns).encode("utf-8")) - hasher.update(str(stat.st_size).encode("utf-8")) - return hasher.hexdigest() - - def _can_use_cached_index(self, fingerprint: str) -> bool: - if not (self.faiss_index_path.exists() and self.metadata_path.exists() and self.state_path.exists()): - return False - try: - state = json.loads(self.state_path.read_text(encoding="utf-8")) - return state.get("docs_fingerprint") == fingerprint - except Exception: - return False - - def build_or_load(self) -> None: - self.index_dir.mkdir(parents=True, exist_ok=True) - fingerprint = self._compute_docs_fingerprint() - - if self._can_use_cached_index(fingerprint): - self.index = faiss.read_index(str(self.faiss_index_path)) - with open(self.bm25_index_path, "rb") as f: - self.bm25 = pickle.load(f) - self.metadata = json.loads(self.metadata_path.read_text(encoding="utf-8")) - self.docs_loaded = len(self.metadata) > 0 - return - - docs = load_documents(self.docs_dir) - chunks = chunk_documents(docs, self.chunk_size_tokens, self.chunk_overlap_tokens) - if not chunks: - self.index = None - self.metadata = [] - self.docs_loaded = False - return - - vectors = self.embedding_service.encode([c["text"] for c in chunks]) - dim = vectors.shape[1] - index = faiss.IndexFlatIP(dim) - index.add(vectors) - - tokenized_corpus = [c["text"].lower().split() for c in chunks] - bm25 = BM25Okapi(tokenized_corpus) - - self.index = index - self.bm25 = bm25 - self.metadata = chunks - self.docs_loaded = True - - faiss.write_index(index, str(self.faiss_index_path)) - with open(self.bm25_index_path, "wb") as f: - pickle.dump(bm25, f) - self.metadata_path.write_text(json.dumps(chunks, ensure_ascii=False, indent=2), encoding="utf-8") - self.state_path.write_text( - json.dumps({"docs_fingerprint": fingerprint, "chunk_count": len(chunks)}, indent=2), - encoding="utf-8", - ) - - def search(self, query: str, top_k: int = 4) -> List[Dict[str, str]]: - if self.index is None or not self.metadata or self.bm25 is None: - self.last_retrieved = [] - return [] - - # Vector Search (FAISS) - query_vec = self.embedding_service.encode_query(query) - faiss_scores, faiss_indices = self.index.search(np.asarray(query_vec, dtype=np.float32), top_k * 2) - - # BM25 Search - tokenized_query = query.lower().split() - bm25_scores = self.bm25.get_scores(tokenized_query) - - # Get top indices for BM25 - bm25_top_indices = np.argsort(bm25_scores)[::-1][:top_k * 2] - - # Combine using Reciprocal Rank Fusion (RRF) - # RRF_score = 1 / (k + rank) - k = 60 - rrf_scores = {} - - # Add FAISS ranks - for rank, idx in enumerate(faiss_indices[0]): - if idx < 0 or idx >= len(self.metadata): - continue - rrf_scores[idx] = rrf_scores.get(idx, 0.0) + (1.0 / (k + rank + 1)) - - # Add BM25 ranks - for rank, idx in enumerate(bm25_top_indices): - if idx < 0 or idx >= len(self.metadata): - continue - rrf_scores[idx] = rrf_scores.get(idx, 0.0) + (1.0 / (k + rank + 1)) - - # Sort by combined RRF score - sorted_indices = sorted(rrf_scores.keys(), key=lambda x: rrf_scores[x], reverse=True)[:top_k] - - results: List[Dict[str, str]] = [] - for idx in sorted_indices: - chunk = self.metadata[idx] - # Fetch the original FAISS score for fallback relevance checking if needed, or just use RRF score - # Note: RRF scores are small (e.g. 0.03), so we must adjust the threshold in rag_pipeline - results.append( - { - "id": chunk["id"], - "source": chunk["source"], - "text": chunk["text"], - "score": float(rrf_scores[idx]), # RRF score - } - ) - self.last_retrieved = results - return results - - def health(self) -> Dict[str, object]: - return { - "docs_loaded": self.docs_loaded, - "index_ready": self.index is not None, - "chunk_count": len(self.metadata), - "index_path": str(self.faiss_index_path), - } \ No newline at end of file diff --git a/backup_2026-05-04/app/ui_gradio.py b/backup_2026-05-04/app/ui_gradio.py deleted file mode 100644 index 426bf5d006e060e8c5c94324fb2605019c24f485..0000000000000000000000000000000000000000 --- a/backup_2026-05-04/app/ui_gradio.py +++ /dev/null @@ -1,275 +0,0 @@ -import asyncio -import logging -import os -import random -import string -import traceback -import uuid -from typing import Dict, List, Tuple - -from app.services import session_store as _ss - -import gradio as gr -import httpx - -from app.core.config import get_settings -from app.services.embeddings import EmbeddingService -from app.services.llm import LLMService -from app.services.rag_pipeline import RAGPipeline -from app.services.vector_store import FaissVectorStore -from app.services.reranker import RerankerService - -logger = logging.getLogger(__name__) -settings = get_settings() -API_URL = os.getenv("RAG_API_URL", "").strip() - -# Initialize services -embedding_service = EmbeddingService(settings.embedding_model) -vector_store = FaissVectorStore( - embedding_service=embedding_service, - docs_dir=settings.docs_dir, - index_dir=settings.index_dir, - chunk_size_tokens=settings.chunk_size_tokens, - chunk_overlap_tokens=settings.chunk_overlap_tokens, -) -llm_service = LLMService( - provider=settings.llm_provider, - groq_api_key=settings.groq_api_key, - groq_model=settings.groq_model, - groq_rewrite_model=settings.groq_rewrite_model, - hf_api_key=settings.hf_api_key, - hf_model=settings.hf_model, - timeout_s=settings.request_timeout_s, -) -reranker_service = RerankerService(settings.reranker_model) -pipeline = RAGPipeline( - vector_store=vector_store, - llm_service=llm_service, - reranker=reranker_service, - top_k=settings.top_k, - max_context_chunks=settings.max_context_chunks -) - -# Ensure index is ready -vector_store.build_or_load() - - -# ── Keyboard layout for realistic typos ────────────────────────────── -NEARBY_KEYS = { - 'a': 'sqwz', 'b': 'vghn', 'c': 'xdfv', 'd': 'sfecx', 'e': 'wrsdf', - 'f': 'dgrtcv', 'g': 'fhtybn', 'h': 'gjyunb', 'i': 'uojkl', 'j': 'hkunmi', - 'k': 'jlomi', 'l': 'kop', 'm': 'njk', 'n': 'bhjm', 'o': 'ipkl', - 'p': 'ol', 'q': 'wa', 'r': 'edft', 's': 'awedxz', 't': 'rfgy', - 'u': 'yihj', 'v': 'cfgb', 'w': 'qase', 'x': 'zsdc', 'y': 'tghu', - 'z': 'xas', -} - -TYPO_CHANCE = 0.07 # 7% chance per word - - -def _nearby_char(ch: str) -> str: - """Return a plausible neighbouring key for the given character.""" - lower = ch.lower() - if lower in NEARBY_KEYS: - replacement = random.choice(NEARBY_KEYS[lower]) - return replacement.upper() if ch.isupper() else replacement - return random.choice(string.ascii_lowercase) - - -def _to_history(messages: List[Dict[str, str]]) -> list: - """Returns the history as a list of dicts (already in this format for Gradio 5).""" - return messages - - -async def _chat_via_api(message: str, history: List[Dict[str, str]]) -> str: - payload = {"message": message, "history": history} - headers = {"Content-Type": "application/json"} - if settings.api_key: - headers["x-api-key"] = settings.api_key - - async with httpx.AsyncClient(timeout=30.0) as client: - resp = await client.post(API_URL, json=payload, headers=headers) - if resp.status_code == 429: - return "⚠️ I'm receiving too many requests right now. Please wait a moment before sending another message." - resp.raise_for_status() - return resp.json()["reply"] - - -async def _get_reply(message: str, chat_history: List[Dict[str, str]]) -> str: - """Fetch the full reply from the LLM (API or direct pipeline).""" - if API_URL: - return await _chat_via_api(message, chat_history) - result = await pipeline.chat(message=message, history=chat_history) - return result["reply"] - - -async def chat_fn(message: str, chat_history: List[Dict[str, str]], session_id: str = ""): - """Streaming generator: yields word-by-word with human-like timing & typos.""" - if not message or not message.strip(): - yield gr.update(interactive=True), chat_history - return - - original_history = chat_history.copy() - chat_history = chat_history + [ - {"role": "user", "content": message}, - {"role": "assistant", "content": ""} - ] - yield gr.update(value="", interactive=False), chat_history - - try: - reply = await _get_reply(message, original_history) - except httpx.HTTPStatusError as e: - logger.error(f"HTTP Error: {e.response.status_code} - {e.response.text}") - reply = "⚠️ Rate limit reached. Please slow down a bit!" if e.response.status_code == 429 \ - else "⚠️ I encountered an error. Please try again in a few seconds." - except Exception as e: - logger.error(f"Unexpected error: {str(e)}\n{traceback.format_exc()}") - reply = f"⚠️ Oops! Something went wrong." - - # ── Humanistic letter-by-letter typing — HTML-safe ──────────────────── - displayed = "" # what is shown in the chat bubble - buffer = "" # invisible accumulator for mid-HTML-tag characters - in_tag = False # True while we are inside a < … > sequence - - for char in reply: - if char == '<': - in_tag = True - buffer += char - continue - - if in_tag: - buffer += char - if char == '>': - # Tag is now complete — flush it to the display all at once - in_tag = False - displayed += buffer - buffer = "" - chat_history[-1] = {"role": "assistant", "content": displayed} - yield gr.update(value="", interactive=False), chat_history - # Small pause after a
to mimic line-break pacing - if displayed.endswith('
') or displayed.endswith('
'): - await asyncio.sleep(random.uniform(0.1, 0.2)) - continue - - # ── Normal character (not inside a tag) ─────────────────────────── - # Decide typo chance only for plain alphabetic words - do_typo = ( - char.isalpha() - and len(displayed) > 8 # skip the very beginning - and random.random() < TYPO_CHANCE - ) - - if do_typo: - wrong_char = _nearby_char(char) - displayed += wrong_char - chat_history[-1] = {"role": "assistant", "content": displayed} - yield gr.update(value="", interactive=False), chat_history - await asyncio.sleep(random.uniform(0.15, 0.3)) - - # Backspace the wrong character - displayed = displayed[:-1] - chat_history[-1] = {"role": "assistant", "content": displayed} - yield gr.update(value="", interactive=False), chat_history - await asyncio.sleep(random.uniform(0.1, 0.2)) - - # Type the correct character - displayed += char - chat_history[-1] = {"role": "assistant", "content": displayed} - yield gr.update(value="", interactive=False), chat_history - - # Pacing: punctuation pauses, space micro-pause, normal char speed - if char in '.!?': - await asyncio.sleep(random.uniform(0.35, 0.7)) - elif char in ',:;': - await asyncio.sleep(random.uniform(0.15, 0.3)) - elif char == ' ': - await asyncio.sleep(random.uniform(0.06, 0.13)) - else: - await asyncio.sleep(random.uniform(0.02, 0.06)) - - # ── Fire-and-forget: persist to session store (never blocks the UI) ─── - # We only save successful responses, not error messages - if session_id and not displayed.startswith("⚠️"): - asyncio.create_task(_ss.save_message(session_id, message, displayed)) - - # Re-enable the input box at the end - yield gr.update(interactive=True), chat_history - - -custom_css = """ -/* Aggressively force height: auto and min-height: 40px on the chatbot and flex containers */ -.gradio-container, .flex, .block, #chatbot-window { - height: auto !important; - min-height: 40px !important; -} - -/* Ensure the chatbot doesn't overflow the viewport if it gets too full */ -#chatbot-window { - max-height: 70vh !important; - border: none !important; -} - -/* Aggressively hide the footer, including HF injected footers if inside the container */ -footer, .footer, footer * { - display: none !important; - visibility: hidden !important; - height: 0 !important; - margin: 0 !important; - padding: 0 !important; -} -""" - -# JS: runs immediately on page load — generates/loads session_id from localStorage -# and restores previous chat history. Completely non-blocking. -_SESSION_JS = """ -async () => { - // ── Session ID: generate once, persist forever in localStorage ── - let sid = localStorage.getItem('martech_session_id'); - if (!sid) { - sid = 'sess-' + Date.now() + '-' + Math.random().toString(36).slice(2, 9); - localStorage.setItem('martech_session_id', sid); - } - // Write the session_id into the hidden Gradio textbox - const el = document.getElementById('session-id-box')?.querySelector('textarea'); - if (el) { el.value = sid; el.dispatchEvent(new Event('input', {bubbles:true})); } - return sid; -} -""" - -with gr.Blocks(title="Martechsol Assistant", theme=gr.themes.Soft(), css=custom_css) as demo: - gr.Markdown("# Martechsol Assistant") - gr.Markdown("Welcome! How can I help you today?") - - chatbot = gr.Chatbot( - show_label=False, - elem_id="chatbot-window", - render_markdown=True, - ) - with gr.Row(): - msg = gr.Textbox( - placeholder="Type your question here...", - show_label=False, - scale=9 - ) - send = gr.Button("Send", variant="primary", scale=1) - - clear = gr.Button("Clear Chat History") - - # Hidden textbox holds the session_id — invisible to the user - session_id = gr.Textbox( - value="", - visible=False, - elem_id="session-id-box", - label="session_id", - ) - - # On load: run JS to set session_id from localStorage (non-blocking) - demo.load(fn=None, inputs=None, outputs=session_id, js=_SESSION_JS) - - # Wire up the events — streaming generator for live typing effect (UNCHANGED) - send.click(chat_fn, inputs=[msg, chatbot, session_id], outputs=[msg, chatbot]) - msg.submit(chat_fn, inputs=[msg, chatbot, session_id], outputs=[msg, chatbot]) - clear.click(lambda: [], None, chatbot) - -if __name__ == "__main__": - demo.launch(server_name="0.0.0.0", server_port=7860, show_api=False) diff --git a/backup_2026-05-04/docker-compose.yml b/backup_2026-05-04/docker-compose.yml deleted file mode 100644 index 24d412fb9c78ed97ae3106fa07ef069bb5679c95..0000000000000000000000000000000000000000 --- a/backup_2026-05-04/docker-compose.yml +++ /dev/null @@ -1,14 +0,0 @@ -version: "3.9" -services: - rag-api: - build: . - container_name: rag-api - ports: - - "8000:8000" - env_file: - - .env - volumes: - - ./docs:/app/docs - - ./data/index:/app/data/index - - ./data/sessions:/app/data/sessions - restart: unless-stopped diff --git a/backup_2026-05-04/docs/MartechSol_Employee_HandBook (7) (1).pdf b/backup_2026-05-04/docs/MartechSol_Employee_HandBook (7) (1).pdf deleted file mode 100644 index ae104106fb1a7f1745db014121416ba1ec57054c..0000000000000000000000000000000000000000 --- a/backup_2026-05-04/docs/MartechSol_Employee_HandBook (7) (1).pdf +++ /dev/null @@ -1,3 +0,0 @@ -version https://git-lfs.github.com/spec/v1 -oid sha256:a48b14cc28d7892ae03a5dfe06a244832e2807ce178ca1f1acdbc58d6af4e8ef -size 3844502 diff --git a/backup_2026-05-04/docs/MartechSol_Employee_Handbook_Extracted.txt b/backup_2026-05-04/docs/MartechSol_Employee_Handbook_Extracted.txt deleted file mode 100644 index 5035befdbf23a1f6638cd933f3f855043175d423..0000000000000000000000000000000000000000 --- a/backup_2026-05-04/docs/MartechSol_Employee_Handbook_Extracted.txt +++ /dev/null @@ -1,1091 +0,0 @@ -Employee Handbook -Last Modified: September 1st, 2024 - -Contents -Introduction ....................................................................................................... 4 -Workplace Orientation ....................................................................................... 4 -Employment Status .......................................................................................... 5 -Employment At Will .......................................................................................... 5 -Employee Responsibilities ........................................................................................ 5 -Workweek and Workday .................................................................................... 5 -Time Clocks and Time Cards ............................................................................... 5 -General Attendance Requirements ....................................................................... 6 -Work from Home .............................................................................................. 6 -Problem Resolution ........................................................................................... 7 -Personal Appearance ......................................................................................... 7 -Health Safety .................................................................................................. 7 -Employee Relationships ..................................................................................... 8 -Policy Against Harassment ................................................................................. 8 -Activity Permission ........................................................................................... 9 -Confidentiality ................................................................................................ 9 -Intangible Property ......................................................................................... 10 -Restraint of Trade, Non-Compete, and Cooling Off Period ....................................... 11 -Solicitation ................................................................................................... 11 -Cell Phone Usage ........................................................................................... 12 -Computer, E-Mail and Internet Policy ................................................................. 12 -Tobacco Use .................................................................................................. 13 -Grievance Procedure ....................................................................................... 13 -Copyright Violation ......................................................................................... 13 -Data Theft .................................................................................................... 13 -Separation from the Company .......................................................................... 14 -Exit Interviews .............................................................................................. 14 -Final Settlement ............................................................................................. 14 -Return of Property .......................................................................................... 15 - - -Acceptability ................................................................................................. 15 -Work Rules ................................................................................................... 15 -Level 1 Actions .............................................................................................. 16 -Level 2 Actions .............................................................................................. 16 -Level 3 Actions .............................................................................................. 17 -Disciplinary Procedures ..................................................................................... 18 -Employee Benefits ................................................................................................. 18 -Referral Program ............................................................................................ 18 -Game Room .................................................................................................. 18 -Holidays and Holiday Pay ................................................................................. 18 -Pay Period and Paychecks ................................................................................ 19 -Leaves ......................................................................................................... 19 -Sick Leaves ................................................................................................... 19 -Casual Leaves ............................................................................................... 20 -Annual Leaves ............................................................................................... 21 -Hajj Leave .................................................................................................... 22 -Maternity Leave ............................................................................................. 23 -Paternity Leave .............................................................................................. 23 -Bereavement Leave ........................................................................................ 24 -Unauthorized Leaves ....................................................................................... 24 -Unapproved Absence Without Pay ..................................................................... 25 -Leave Encashment ......................................................................................... 25 -Other Benefits ..................................................................................................... 25 -Loans & Advance Salary .................................................................................. 25 -Maternity (Pregnancy) Benefit .......................................................................... 26 -Company Maintained Vehicle ............................................................................ 27 -Healthcare Insurance – For Self & Spouse ............................................................ 27 -EOBI: .......................................................................................................... 30 -Provident Fund: ............................................................................................. 30 - - -Introduction -We are providing you this employee handbook to introduce you to our policies. The handbook will -describe some of the more important employee benefits and obligations. -Our business is constantly changing, and therefore MartechSol reserves the right to change all of our -policies, including those covered in this handbook, at any time. You will be informed of any changes -in policy through posting on portal or through other appropriate means. If you are uncertain about -any policy or procedure, please check with your Supervising Authority. -Workplace Orientation -Orientati -on Training and Initial Evaluation Period -All employees will receive orientation training at the beginning of employment with MartechSol. This -period of orientation will include familiarization with Logicose’s work policies, disciplinary policies, job -duties, and reporting responsibilities. -Upon completion of orientation, all employees will undergo evaluation during the first ninety (90) -calendar days of employment. If performance is unsatisfactory for any reason during this period of time, -the employee may be immediately dismissed from employment. At the end of this 90-day period, the -employee’s Supervising Authority will determine if the employee’s performance warrants continued -employment. -If the employee’s performance is not up to the Supervising Authority’s expectations in terms of quality of -work, output, behavior, or any other performance indicator, the employee will not be confirmed for -further employment at the company after this 90-day period. It is also up to the Supervising Authority to -end the probation period at any point before 90 days and either confirm or terminate the employee. -The Supervising Authority is required to make a decision regarding the confirmation of an employee after -90 days. If the Supervising Authority requires more time to evaluate an employee further, (s)he has the -right extend the probation period. -Confirmation -If performance exceeds expectations, an employee may receive a salary increment upon confirmation. -The date of confirmation depends on date of joining and performance. Revised salaries for employees -that are confirmed before the 15th of a given month will be effective from the 1st of that same month. On -the other hand, revised salaries will be effective from the 1st of the following month for employees that -are confirmed after the 15th of a given month. For example, if someone is confirmed on the 10th of April, -the revised salary will be effective from the 1st April, however if they are confirmed on the 16th of April, -the revised salary will be effective from the 1st of May. -Casual and sick leaves will also be assigned on a pro-rata basis based on this calculation. - - -Employment Status -A “regular full-time employee” is any employee who is employed to work on a year-round basis, and who -is regularly scheduled to work their full shift hours. -Employment At Will -All employees are employed on an "at-will" basis. This means that employees are not employed for any -specific term or period of time, and that the employer may terminate the employment relationship at -any time for any reason. Similarly, employees may resign and terminate employment with MartechSol -at any time and for any reason with prior notice of 30-90 days depending on their role within the -organization. -N -o employee has any authority to change the nature of this relationship through any statements, -representations, or conduct. The at-will relationship may be changed only through written and signed -authorization of the Supervising Authority. -Employee Responsibilities -Workweek and Workday -The nor -mal workweek shall be considered full shift hours, six days per week, from 9:00 a.m. to 5:30 p.m. -The first Saturday of every month is off and the remaining three Saturday’s in each month are six and a -half hours (half-day). One hour is allocated for lunch/dinner in all shifts. - Morning Shift: 1:30 pm - 2:30 pm - Evening Shift: 9:30 pm - 10:30 pm -Smoking Breaks: There are a maximum of 3 breaks allowed in a workday (other than lunch/dinner -break). These breaks are not to be longer than 5 minutes. -If any other break is required, the employee is to contact his or her Supervising Authority for approval. -Time Clocks and Time Cards -All empl -oyees shall use the portal to record their starting time and ending time. For wage payment -purposes, time worked will be figured to the nearest tenth of an hour. -Sharing login information (username and password) is strictly prohibited. Each person is to use their own -login information to time in at work. If any discrepancy is observed in the time in or time out process, -harsh disciplinary action will be taken against the person or persons involved. -In case an employee forgets to time in or is unable to time in, they are to submit a trouble ticket to the -Human Resources department. It is the employee’s responsibility to maintain their time sheet accurately. -Monthly salary will be calculated according to the time sheet. -Employees who want to change their work shift should contact their Supervising Authority for approval. -Necessary paperwork and legitimate reason are required for the shift change process. The decision to -change the shift of an employee is at the discretion of the Supervising Authority. - - -General Attendance Requirements -Regular and pro -mpt attendance of employees is essential to our business. Employees who cannot meet -our attendance requirements will be subject to counseling and disciplinary action. Excessive absenteeism -or tardiness will result in termination of employment. The Supervisor has the authority to mark a half -day, day off, or full day at his/her discretion. - Late Coming: Late coming is permitted till 20 minutes from your shift start time. If you arrive -after 20 minutes of your shift start time, then it will be considered as a late arrival. 4 late arrivals -in a month will be considered as a half day off. In addition, an employee is to inform his/her -Supervising Authority if they will be more than 30 minutes late to work. - Early Out: If you need to leave before your shift ends, verbal approval is mandatory from your -Supervising Authority. There will be no deductions for early out. - Half Day: Half day will be marked if you arrive 2 hours after your shift start time. If you complete -less than six and a half hours on any given day, then it will also be considered as a half day off. - Full Day: Employees who spend less than 2 hours at work on any workday will be marked absent -that day. - Employees who are late to work on any given workday are still responsible to complete all the -work assigned to them. If they are unable to, they should contact their Supervising Authority -immediately. If an employee arrives at work after 1 hour of their shift start time, by default, -the 8.5 hours shift has to be completed. If an employee is more than 1 hour late and wants - to -leave early (without completing the total (8.5 hr) shift time), he/she has to contact their -Supervising Authority for approval. - For -employees who are unable to complete at least 50% of their work during a workday, the -Supervising Authority reserves the right to decide the status of the workday (full day, off day, -half day) is at his or her discretion. -Work from Home -In case o -f emergency (bad weather, temporary disability, unrest in city, etc) when employees cannot -reach work, they will be requested to work from home. In case an employee has personal valid reason(s) -as to why they cannot reach work, they can work from home requesting approval from their Supervising -Authority. The approval of this request is at the Supervising Authority’s discretion. -If less than 30% of the work has been completed, the day will be counted as a full day off. If 30-70% of -the work has been completed, it will be counted as a half day. If more than 70% of the work has been -completed, it will be counted as a full day. -Work can be submitted no later than 3 hours after the end of the employee’s regular shift timing. If the -work is not submitted within the given time, it will not be marked as work provided for the same day. If -an employee faces problems submitting their work within the given duration and wants to submit their -work later, approval is required from their Supervising Authority. - - -Problem Resolution -If an emplo -yee faces difficulty or problems related to anything within the workplace, it is their -responsibility to submit a trouble ticket through the Portal. There is no need to contact the Supervising -Authority unless their ticket has not been addressed for more than 24 hours. -You are requested to use this trouble ticket system for your computer, admin and HR related -issues. How to use this system: -1. If you have any computer, admin, or HR related issue, go to the tickets link in the portal and -select ‘Submit a ticket’. -2. Select the concerned department for your problem e.g. if your mouse, keyboard, CPU or LCD is -not working properly, please select Networking. If there is any issue with your attendance or -payroll, select Human Resources. If there is any issue with fan, AC, kitchen item etc. please select -Admin department. -3. Once your trouble ticket is submitted, you may view the status of your ticket from ‘View open -tickets’ link. -As soon as your ticket is submitted, the Company will make every possible effort to resolve your issue on -an urgent basis. -There is also a messaging feature in this trouble ticket system. If there is any update related to your -ticket, you will receive a message within your open ticket. Similarly, if you want to send any message -related to your problem, you can also do that from ‘View open tickets’ section. -Once your issue is resolved, your ticket will be closed. You may view your closed tickets from ‘View your -closed tickets’ section. -This system will help resolve your issues quickly and will allow you to view status of your open tickets. -Personal Appearance -A company - is judged by its employees, as well as by its products and services. All employees are -expected to maintain good standards of personal neatness, including hair, clothes, and shoes. -Casual to business-style dress is appropriate. Employees should be neatly groomed and clothes should be -clean and in good condition. Leisure clothes such as cut-offs or halter tops are not acceptable attire for -the business office. -Health Safety -If an employee has had a contagious illness or serious health issue which may affect the health of other -employees (e.g. chicken pox, flu), a doctor’s written permission to return to work will be necessary. - - -Employee Relationships -MartechSol employs a number of people and, as a result, you will be working with people whose -personalities may differ from yours. It is your responsibility to get along with all of your co-workers. A -spirit of cooperation shown by all of us will help to accomplish our purpose of providing new -customers with the best possible service. -This does not mean that you need to become a personal friend of all your co-workers, but it does mean -that you should treat your co-workers with respect. Regardless of your personal opinions, your fellow -employee should be treated exactly as you would like to have that person treat you. -Senior employees should remember that they were once new employees and therefore should do -everything possible to aid new employees in acclimating themselves to their new jobs. -Policy -Against Harassment -MartechSol is committed to maintaining a work environment that is free of discrimination and -harassment. In keeping with this commitment, we will not tolerate harassment of our employees by -anyone, including any Supervising Authority, co-worker, vendor, client, or customer. -Harassment consists of unwelcome conduct, whether verbal, physical, or visual, that is based upon a -person’s protected status, such as sex, color, race, ancestry, religion, national origin, age, physical -handicap, medical condition, disability, marital status, veteran status, citizenship status, or other -protected group status. -MartechSol will not tolerate harassing conduct that affects tangible job benefits, that interferes -unreasonably with an individual’s work performance, or that creates an intimidating, hostile, or -offensive working environment. -Sexual harassment deserves special mention. Unwelcome sexual advances, requests for sexual favors, -and other physical, verbal, or visual conduct based on sex constitute sexual harassment when (1) -submission to the conduct is made a term or condition of employment, (2) submission to or rejection of -the conduct is used as the basis for an employment decision, or (3) the conduct has the purpose or effect -of unreasonably interfering with an individual’s work performance or creating an intimidating, hostile, or -offensive working environment. Sexual harassment may include explicit sexual proposition, sexual -innuendo, suggestive comments, sexually oriented “kidding” or “teasing,” “practical jokes,” jokes about -gender-specific traits, foul or obscene language or gestures, displays of foul or obscene printed or visual -material, and physical contact, such as patting, pinching, or brushing against another’s body. -All employees are responsible for helping to assure we avoid harassment. If you feel that you -have experienced or witnessed harassment, you should immediately notify your Supervising -Authority. MartechSol forbids retaliation against anyone who has reported harassment. -It is our policy to investigate all such complaints thoroughly and promptly. To the fullest extent -practicable, MartechSol will keep complaints and the terms of their resolution confidential. If an -investigation confirms that harassment has occurred, MartechSol will take appropriate corrective -action, including discipline up to and including immediate dismissal from employment. - - -Activity Permission -MartechSol is concerned with outside activities of employees only where such activities might involve a -conflict of interest with Logicose’s best interests. A conflict of interest exists if an employee has a -private financial interest or other relationship outside MartechSol that is detrimental to the best -interests of MartechSol. -You shall not without prior written consent of the company, engage directly or indirectly, in any trade, -business or occupation or any other income generation activity. -If an employee desires to attend an academic institution during his/her employment, (s)he must receive -permission from the company by submitting relevant documentation. It is mandatory to fill the Activity -Permission form for approval. -An employee should discuss any possible conflict of interest with the HR Department as full disclosure -and open knowledge are essential. For the employee’s own protection, written approval should be -obtained in all cases where a possible conflict of interest is involved. -Other possible conflict of interest situations may include any type of “moonlighting work” or a second -job, which interferes with the employee’s work for MartechSol, any type of external business -relationship with MartechSol, any of its competitors or any of its customers, and any type of public -activity which may affect adversely on MartechSol. -Confidentiality -MartechSol is a service provider that creates, completes, and delivers work for clients that do not want -our involvement disclosed in their projects. The ownership of the work we create for our clients is -transferred over to them and they hold the copyright to the work delivered. Disclosing such work may -result in Copyright Infringement, so it is in your, and the company’s best interest, to not share -involvement in such work. -Moreover, employees are prohibited from maintaining records of confidential information or intellectual -property on personal e-mail accounts, computers, hard drives, smart phones, cloud, or online. -Both during employment with MartechSol and after separation, employees are not permitted to -disclose to any third party or share any confidential information that is made available to you. -Confidential information includes, but is not limited to: - Web copy, eBooks, Press Releases, Reports, Infographics, Articles, Blogs, and/or any copy that was -created or revised for clients while employed at MartechSol is the property of either the clients or -MartechSol. Disclosing involvement in such projects to those outside the company, mentioning it -on your CV or resume, and/or making public your involvement in such projects is prohibited. - Search Engine Optimization strategy, campaigns, processes, proposals, accounts for posting, list -of clients, contact information of clients, content created or revised for client is confidential -information. Mentioning your involvement and/or disclosing such information to those outside -the company or on your CV or resume is considered breach of policy. - Client Information, leads, pricing strategy, proposals, account information, portal information, -samples, and promotional campaigns are the intellectual property of MartechSol. They may -neither be disclosed nor shared with those outside the organization. - - - Sensitive company information, such as employee contact details, software code, promotional -campaigns, client list, project(s) details, performance evaluation system, and in-house software. - Any and all such information is either Logicose’s or its Client’s intellectual property. -Immediately upon termination of employment, or at any other time upon the company’s request, all -employees will return to the Company: - All memoranda, notes and data, and computer software and hardware, records or other -data/documents compiled by the employee or made available to the employee during the -employee’s employment with the company concerning the business of the company, its -affiliates, or their customers. - All other Confidential Information that is disclosed to you during employment. - All physical and intellectual property of the company or its affiliates, including without limitation, -all the content, information, plans, files, records, documents, lists, equipment, supplies, -promotional materials, keys, vehicles, and similar items and all copies thereof or extracts there -from. -Disclosure of such information while employed or after separation from MartechSol is unethical, and -such actions will be considered intellectual property theft. -Furthermore, you cannot claim ownership of any project or developed content/material in any manner. -For example, giving a reference on your CV to a specific website for which you developed content while -working at the Company is prohibited. Any project or client details relating to your work carried out at -this company cannot be mentioned on your CV. -Intangible Property -During your course of employment with MartechSol, you will come across information that is privy to -those outside the organization. Such information is the intellectual (intangible) property of the company. -You may not at any time, during or after employment with MartechSol, have or claim any right, title, or -interest in any trade name, trademark, patent, copyright, work for hire, or other similar rights belonging -to or used by the company and shall not have or claim any right, title, or interest in any material or -matter of any sort prepared for or used in connection with the business or promotion of the company, -whatever your involvement with such matters may have been, and whether procured, produced, -prepared, or published in whole or in part by you. -MartechSol, or its clients, have and will retain the sole and exclusive rights in any and all such trade -names, trademarks, patents, copyrights, material, and matter unless transferred over to a third party. -Any and all documents, software, or copy that was created for MartechSol, its clients, or its affiliate -brand names, is the intellectual property of the company or its clients. You may not claim ownership, -maintain personal records of, or disclose your involvement in the creation or modification of such -information, software, or projects. - - -Restraint of Trade, Non-Compete, and Cooling Off Period -Any current or separated employee is prohibited from incorporating a new company or starting a new -business in the same industry before the time of at least 24 months. -Employee shall not, on his or her own behalf or behalf of others, hold any ownership interest in any -business engaged in the following industries: - Copywriting - Online Marketing - SEO - SMO - Ghostwriting - Industries that MartechSol, or any of its brand names, are currently or in the process of operating in -Employee agrees and covenants that the use of sensitive and confidential information that is made available, -in certain circumstances, may cause irreparable damage to MartechSol and its reputation. Therefore, -employee shall not until the expiration of two years after the termination of the employment, -through any corporations or associates in any business, form a business that is directly competitive with -Company for a period of two years. -Since the region or territory in which MartechSol operates is based online, the jurisdiction of the -restraint of trade is not limited to a specific city, state, or country. Forming an organization that is based -in any city MartechSol operates in, and/or creation of an online business is a breach of policy. -Failure to comply with activity permission, restraint of trade, confidentiality, intangible property, non- -disclosure, and non-compete will result in immediate termination. The Company also reserves the right -to take appropriate legal action upon breach of policy both during employment and after separation. -Solicitation -Both during employment with the company and for a period of two (2) years following the termination/ -resignation of employment with the company at any time and for any reason, employee’s will not, directly -or indirectly, on the employee’s own behalf or on behalf of any other person or entity, hire or solicit to -hire for employment or consulting other provision of services, any person who is actively employed or -engaged (or in the preceding six months was actively employed or engaged) by MartechSol. -This includes, but is not limited to, inducting or attempting to induce, or influencing or attempting to -influence, any person employed or engaged by the company to terminate his or her relationship with the -company. MartechSol reserves the right to take appropriate legal action against solicitation of employees. -Furthermore, both during the employee’s employment and after separation from the company at any -time and for any reason, the employee will not directly or indirectly, on the employee’s own behalf of -any other person or entity, solicit the business of any Customer or any other entity with which -MartechSol has worked with or discussed the possibility of any work, at the time of employee’s -termination, to provide services to such entity (a “Contractor”). Client information is confidential, and -MartechSol reserves the right to take legal action upon disclosure of privy information. - - -Cell Phone Usage -Emp -loyees shall not use cell phones for personal purposes while engaged in work activities. If the -employee needs to use the cell phone for important personal reasons, the employee shall excuse -themselves from the premises. Keep your cell phone on silent/vibrate. Use it responsibly, and make sure -that you do not disturb others while answering your calls. -Computer, E-Mail and Internet Policy -MartechSol maintains certain policy requirements for use of its “electronic communication systems.” -“Electronic communication systems” include computers, file servers, electronic information storage -systems, internal and external electronic mail systems, voice mail systems, local area networks and -any other system or technology used now or in the future by MartechSol to store or transmit data of -any kind electronically. -Our - policy requires the following: -1. Employees shall use the electronic communications systems for business purposes. -2. Logicose’s electronic communications systems and all information transmitted or received by, or -stored or contained in, these systems are the property of MartechSol. No employee shall have -any property rights in or expectation of privacy for any information, even if of a personal nature, -communicated or received through Logicose’s electronic communications systems. MartechSol -may review, delete and copy any such information without first obtaining the consent of the -employee who created, sent or received the information. No employee, including system -administrators and Supervising Authority, shall use any electronic communications systems for -purposes of satisfying idle curiosity about the affairs of others. There must be a business purpose -for obtaining access to the files or communications of others. -3. No employee shall use the electronic communications systems in a manner that constitutes a -violation of provincial or federal law or that is offensive to others. -4. Due to the threat to system stability posed by computer viruses and system incompatibility, no -employee shall install or download software from the Internet or elsewhere on his or her -computer without the supervision of the office administrator. -If you have any questions about the policy or its administration, please contact your Supervising -Authority. Violations of the policy may result in disciplinary action up to and including termination. -USBs cannot be used on any office computer. If necessary, please contact the System Administrator. -The completion of assigned work is the employee’s responsibility. If an employee has any issues -regarding their PC or internet, they are to raise a trouble ticket immediately. -The use of the following is prohibited: - Social media websites (e.g. Facebook) - Chatting websites (e.g. Meebo) - Chatting applications (e.g. MSN Messenger, Skype) - Online gaming websites (e.g. miniclips.com) - E-mail sites (e.g. Hotmail) - - - Political Websites - Pornography -Tobacco Use -MartechSol maintains a smoke-free workplace for all of its employees, which is why smoking is -prohibited in the workplace. Employees who must smoke are encouraged to do so away from Company -property and during work and/or meal breaks only. -Grievance -Procedure -MartechSol has a grievance procedure to make sure that problems and complaints an employee has -concerning his or her treatment or working conditions is called to our attention. This does not -include computer-related issues or similar technical issues. For such problems, the trouble ticket -system is provided. Employees with a problem should take the following steps: -1. Explai -n the problem to your immediate Supervising Authority. Allow 48 hours for an answer. -2. If you are still not satisfied, put your grievance in writing and submit it to MartechSol President via -the Feedback Form on the Portal. If possible, a decision will be given within three working days. -Copyright Violation -Any work performed within the premises of MartechSol for its clients, employees, or the Company itself -is under the sole ownership of MartechSol or those that the company transfers the rights to. Employees -cannot claim ownership of this work or use it outside company premises for other purposes, this -includes creating personal records on computers or e-mail accounts that aren’t the property of -MartechSol. Employees cannot use this work as reference to anyone else either, unless they get written -consent from MartechSol. Listed below are some examples of copyright violation: -Showing, sharing, using, creating a personal record of, or claiming ownership or involvement in the -creation of: - Co -py written - Software developed - Client contact information - Payments received at MartechSol - In -tellectual property -Data Theft -Any data relating to MartechSol, its clients, or employees is under the sole ownership of MartechSol. -Employees cannot use or transmit this data, as it is not under their ownership. Employees cannot use -this data as reference either. -Archiving such work on external hard drives, online, cloud, and/or on computers that are not property -of MartechSol is a breach of policy. - - -Separation from the Company - Prio -r to Confirmation: -o Minimum one day notice period, or as communicated in the offer letter, is required -during probationary period on behalf of the employee. -o If employee performance/behavior is not up to Company expectations, then the -Supervising Authority has the right to end the probation period and terminate the -employee. - After Confirmation: -o The employment relationship may be terminated either by you or by the company with a -prior written notice of one month or salary in lieu thereof, unless mentioned below. It is, -however, understood that no notice is required in case your services are terminated on -grounds of misconduct, willful neglect of duty, breach of trust or any other dereliction of -duty or misdemeanor prejudicial to the interest of the company. It is further understood -that any financial indebtedness of yours, standing to the detriment of the company’s -prestige will amount to a very serious misdemeanor towards the company -o A minimum notice period of at least 30 days is to served unless listed below otherwise: -Managerial Roles -Vice Team Lead 30 days -Team Lead 60 days -Group Team Lead 90 days -Exit Interviews -All employees separating from MartechSol must complete an exit interview with their immediate -supervisor. An exit interview gives the Human Resource Department the opportunity to obtain feedback -from employees as they transition out of their positions. -Final Settlement -All employees must undergo a final settlement before they separate from the organization. All -adjustments will be made according to below mentioned procedures: - Bef -ore Salary is Processed: -If any employee resigns before the salary is processed of any given month, their adjustments -(loans, advances, pro-rata leaves, etc.) will be made in the same month and the salary will be -deposited along with the rest of the employees. The remaining amount of the following month’s -salary will be processed as per the payroll process. No adjustments will be made in the following -salary except for early exit. For example, someone resigns on the 20th of April and his last working -day is the 20th of May, the adjustments will be made in the month of April’s salary. The salary for -the remaining days of the notice period to be served in May will be processed with May’s payroll -with Early Exit deduction of 11 days (21st May-31st May). - After/At the time Salary is Processed: -If any employee resigns once the salary is processed of any given month, their adjustments (loans, -advances, pro-rata leaves, etc.) will be made in the following month and the salary will be -deposited along with the rest of the employees. For example, someone resigns on 25th of April and - - -his last working day is on the 25th of May, the adjustments will be made in the month of May’s -salary. -Return of Property -Employees who separate from MartechSol service must return all issued property such as identification -cards, health insurance card, company maintained vehicle, and intellectual property of the organization. -Acceptability -Any changes in Policies, Rules, and Code of Conduct will first be shared through the MartechSol Portal. -A reasonable period of time will be allotted to discuss suggestions and amendments in changes before -its implementation. -Timing In to work after date of implementation implies that you understand and will abide by -organization policy. -Work -Rules -Any work performed within the premises of MartechSol is under the Company’s ownership and/or its -clients. Employees cannot claim ownership of any work done in or for MartechSol, neither can they use -it in the future. -All work performed at MartechSol has to be done in a legal and ethical manner so there is never -any copyright violation. -Employees shall perform their assignments within the specifics of the position description. Employees -who consistently fail to conform to the specifics of their position description or exhibit inappropriate -behavior or poor performance shall be required to meet with their supervisor. -This meeting will attempt to identify the problems, find ways to improve the situation and suggest -adequate solutions, concluding with a recommended course of action and an appropriate time frame -in which the employee will be expected to improve to the satisfaction of MartechSol. Details of the -meeting will be documented, signed by all parties as a correct representation of points discussed and -placed in the employee’s personal file. -Disciplinary action and corrective measures are taken at the discretion of the Management at -MartechSol. The possible steps for disciplinary action are: -1. Co -unseling/Verbal Warning -2. One or more formal written warnings through email -3. Issuance of Notice -4. Suspension -5. Dismissal -The choice of options depends on the seriousness of the behavior. Exceptions or deviations from the -progressive discipline sequence may occur whenever the Supervising Authority, in conjunction with the -President/Director, deems that circumstances warrant. -For level 1 offense, discussions between the employee and his or her supervisor will occur to allow the -employee to correct the situation. When a warning notice is issued, it becomes a part of an employee's - - -record and is considered when evaluating an employee for promotion, transfer, training or additional -discipline. Various offenses have been divided into different levels according to severity. -Three warning notices within twelve (12) months’ time, regardless of the type of level offense, will result -in discharge. -Level 1 Actions -These actions are taken for behaviors such as: - Unautho -rized or excessive absence, tardiness or early timing out - Unauthorized time away from work station - Slowing or interfering with the work of other employees - Failure to notify supervisor of incompletion of assigned work before leaving - Obscene, abusive, harassing, or disruptive language or behavior - Failure to perform assigned job responsibilities - Failure to follow prescribed work procedures - Failure to notify supervisor of absences - Neglect of organization property - Excessive personal use of cell telephone and/or email - Work shift not completed -Procedure For Dealing With Level 1 Behavior: These procedures are at the discretion of the Supervising -Authority. - Counseling/Verbal warning - Formal written warning in the form of email - Issuance of notice - Suspension -Level 2 Actions -These are more serious and must be dealt with firmly and immediately. Typical behaviors in this level -include: - Reoccurrin -g tardiness without reasonable explanation - Absences without approved leave - Refusal to comply with instructions of a supervisor - Conduct endangering the safety of the employee, co-workers or members - Working when ability is impaired by the use of alcohol, and/or illegal drugs - Unscheduled leaving from the workplace without informing supervisor - Habitual sleeping during work hours - Unauthorized use of organization materials and supplies - Fighting or threatening violence in the workplace - Unauthorized possession of weapons on organization property - - - Knowingly timing in another employee - Sharing portal login info - Inappropriate usage of internet, or any usage of internet unrelated to work during work hours - Harassing or bullying coworkers -Procedure for dealing with Level 2 behavior: These procedures are at the discretion of the Supervising -Authority. - Issuance of Notice - Suspension or probation - Termination -Level 3 Actions -These are b -ehaviors that are serious enough to justify either a suspension or, in extreme situations, -termination of employment without following the preceding disciplinary steps. Behaviors for which -immediate termination can be justified include, but are not limited to, the following: - Sexual harassment - Involvement in any income-generating activity without the consent of supervising authority - Insubordination, or the refusal to comply with the specific instructions of a supervisor in the -context of an assigned job duty - Falsification of personnel records, time records, or any other organization documents and records - Fighting during work time or on work premises - Use of, or possession of, alcohol or illegal drugs during work time or on work property - Damaging, defacing, or misusing organization property or the property of coworkers - Theft, misappropriation, embezzlement, unauthorized possession or removal of organization -property or the property of employees or customers - Immoral or indecent conduct which occurs on organization property - Possession of explosives, firearms, or other dangerous weapons on work premises - Failure to report an absence for a three-day period without a satisfactory explanation - Unauthorized release of work-related data or information - Continued unsatisfactory job performance - Violation of the organization's conflict of interest/ethical standards - Data theft: Stealing any digital or other type of data (e.g. email, company records, client records) -which belongs to MartechSol. - Copyright violation: Showing ownership of specific work performed at MartechSol. Discussion -of clients, websites, revenue generated, payments, or any work-related details outside the -Company’s premises. - Ot -her behaviors that, in the opinion of the Supervising Authority, seriously threaten the well- -being of co-workers. -In case of any ambiguity, please contact your supervising authority. - - -Procedure for dealing with Level 3 behavior: These procedures are at the discretion of the Supervising -Authority. - Suspension - Termination -Disciplinary Procedures -MartechSol may use progressive discipline as a means of enforcing minor violations of the work rules. -Progressive discipline may include steps such as counseling, written warning, and suspension. -Depending on the nature of the misconduct and the position involved, steps in the process may be -omitted or employment may be immediately terminated. The decision whether to apply progressive -discipline in any particular situation will rest within the sole discretion of management. -Examples of misconduct which will usually result in immediate discharge include theft, fighting or -physical assault, insubordination, dishonesty, falsification of time cards, production records, or other -Company documents, possession of alcohol or illegal drugs in the workplace, and disloyalty or breach of -confidentiality. This list is not intended to limit our right to discharge in other situations when we believe -termination of employment is appropriate. -Employee Benefits -Referral Program -Any employ -ee can benefit from the referral program by referring qualified friends or acquaintances. -The employee who has referred an individual is eligible for a PKR. 5,000 bonus once the new -candidate joins the organization. -Game Room -In order to promote a relaxed environment and provide employees with more forms of -entertainment, MartechSol has set up a game room in Room 301. -This game ro -om is accessible during lunchtime, so the morning shift has access to it from 1:30-2:30 pm -and the evening/night shift has access from 9:00-10:00pm. The game room can also be used after work. -Females are to use the game room on Monday, Wednesday, and Friday during lunchtime. Males should -use the game room on Tuesday, Thursday, and Saturday during lunch hour. -Holidays and Holiday Pay -MartechSol observes all Federal holidays as paid holidays. Due to our nature of work, employees -may be asked to work on specific holidays, according to their availability. They will be awarded -compensatory leave allowance or compensatory leave (employee’s choice), which they can utilize at -their convenience. -To qualify for holiday pay, an employee must have worked his or her regularly scheduled shift on the -day preceding the holiday and must work his or her regularly scheduled shift on the day following the -holiday. - - -Pay Period and Paychecks -For pay c -alculation purposes, the workweek (or “pay period”) will commence 12:01 a.m. on Sunday -morning and end at 12:00 p.m. on Saturday evening. Employees will be paid monthly on (payday), for the -preceding month. If payday falls on a holiday, then paychecks will be provided the day before the -holiday. -Employees are expected to immediately report any mistakes in their paychecks. Underpayments will be -corrected as soon as possible. If it is not practical to issue an employee a new paycheck to correct any -overpayment, the overpayment will be deducted from the employee's next paycheck. -Salary deduction for days off will be on “per-day” basis. {(Gross Salary x 12) / 365}. -Payroll for June and December -The salary for June and December will be processed and transferred during the first week of July and -January respectively. This is to ensure that the transition from the current to the revised salary is smooth -and all adjustments for the preceding month are processed correctly. -Leaves -Prior to Confirmation: - No leaves are allowed prior to your confirmation. - In case you skip a day, you will be marked absent, and your salary will be deducted for the day. - Prior approval from your supervising authority is required for situations where you would be -unable to come into work. - If you are absent from work as a result of sickness or any other unforeseeable personal matter, -you are required to notify your supervisory authority or the HR department by telephone or SMS -within two hours of your scheduled shift starting time. -After Confirmation: -Sick Leaves -Definition: -Sick Leave is defined as the period of time an employee is absent from work with full pay as a -result of an illness or injury. -Policy: - All full time confirmed employees are entitled for a maximum of 8 days of sick leave per annum. - Sick leaves cannot be carried forward to the next year and neither can they be encashed. - Each year on the 1st of January, your leave account will be credited with the sick leave -entitlement. - If anyone is confirmed in MartechSol during the year, then his/her sick leave entitlement will be -on a pro-rata basis. - In case of separation from the company during the year, sick leave availed will be adjusted on - - -pro- rata basis from your final salary payment. - If an employee takes sick leave from work starting any day of the week, any weekend or holiday -that falls in between will be counted as a full day sick leave. - Sick leave availed in excess of entitlement will be treated as casual or annual leave (if available -and subject to approval from reporting authority); incase of insufficient casual/annual leave -balance, excess sick leave will be treated as unpaid leave. - During notice period, no sick leave will be entertained even if the leave shows credit balance. -Process and Documentation: - Planned appointments, surgery, or any medical instance which can be deemed as anticipated is -subject to prior approval from your supervising authority. - If you are absent from work as a result of sickness or injury, you are required to notify your -supervisory authority or HR department by telephone or SMS within two hours of your -scheduled shift starting time (or before if possible). Failure to comply with the process may result -in absent without leave. - You must apply for sick leave immediately after returning from your sick leave by submitting -your leave application to the HR department after approval from your supervising authority. - The submission of a medical certificate or relevant documentation when applying for a sick leave -is mandatory, the leave will be considered absent without pay if proper documentation is not -submitted. - Sick leave can only be availed when sick and cannot be claimed for other purposes. Falsifying -information or using sick leave for a purpose other than illness or injury will make you liable for -disciplinary action. -Casual Leaves -Definition: -Casual leaves may be availed by employees in cases of urgent or unforeseen contingencies. -These cases include, but are not limited to, situations which may arise on account of unforeseen -circumstances e.g. birth/illness/death in the immediate family or of a close relative. -Any personal reason that prevents you from coming into work can be used for your casual leave. -Policy: - All full time confirmed employees are entitled for a maximum of 10 days casual leave per annum. - Casual leaves cannot be carried forward to the next year. - Casual leaves that have not been exhausted over the calendar year will be subject to encashment -with the paycheck of the following month. - Each year on the 1st of January, your leave account will be credited with the casual leave -entitlement if eligible. - If anyone is confirmed in MartechSol during the year, then his/her casual leave entitlement will -be on a pro-rata basis. - In case of separation from the company during the year, casual leave availed will be adjusted on - - -pro-rata basis from your final salary payment. - If an employee takes casual leave from work starting any day of the week, any weekend or -holiday that falls in between will be counted as a full day casual leave. - Casual leave cannot be taken for 3 consecutive days at any given time. If leave availed is for or in -excess of 3 days, the entire period will be treated as unpaid leave. However, if circumstances -warrant, the department head may approve the first 3 days to be treated as casual leave, and -only the additional days to be treated as annual leaves (if available). - Employees can take up to 2 consecutive casual leaves. If the number of casual leaves is 3 or -more, the whole duration will be treated as unpaid automatically unless approved by the line -manager. - In case of any urgent requirement, your reporting authority may ask you to report to work during -your casual leave. - Combining casual leave with a weekly holiday or a public holiday shall be inadmissible and that -additional leave shall be adjusted as leave without pay. For example, if a public holiday falls on -Friday and any employee takes leave on Saturday, all three days will be counted as casual leave. - During notice period, no casual leave will be allowed even if the leave shows credit balance. -Documentation and Process: - If you are absent from work as a result of any urgent personal responsibilities or any -unforeseeable circumstances, you are required to notify your supervisory authority or HR -department by telephone or SMS within two hours of your scheduled shift starting time (or -before if possible). Failure to comply with the process may result in absent without leave. - You must apply for casual leave immediately after returning from your casual leave by submitting -your leave application to the HR department after approval from your supervising authority. - In cases of planned or known circumstances, an employee must apply for casual leave to the -immediate supervisor for approval in advance. -Annual Leaves -Definition: -The purpose of annual leave is to provide employees with a reasonable period of rest and -a significant break from the workplace in order to maintain a healthy work life balance. -Policy: - All Full-time confirmed employee(s) shall be entitled to 14 days paid annual leave after -completion of one year of service. - Annuals leaves are not subject to encashment, except for TL roles or greater and for the Sales & -Support Department. - Up to 7 annual leaves can be carried forward to the next year, if approved by supervising -authority, for the following reasons:  - Marriage - Obligation outside the country (Hajj, Residence Visa Process, etc.) - Or for any planned reason that warrants extended leave.  - - - Carried forward annual leaves can’t be carried forward for more than a year.  - If any employee completes one year of service at MartechSol during the calendar year, then his/ -her annual leave entitlement will be on a pro-rata basis for that year. - In case of separation from the company during the year, annual leave availed will be adjusted on -pro-rata basis from your final salary payment. - Annual leave may be availed at one time during the year or may be divided and availed at various -times during the year, subject to approval. - Annual leaves will be granted in keeping with the operational exigencies of the organization, and -will be scheduled by the supervisor to meet the requirements of individual employees as far as -possible. - Management reserves the right to refuse a leave request, allow a partial request, revoke -approval if already granted, and/or to recall an employee before expiry of the leave period. - If you are hospitalized during your annual leave and produce a valid medical certificate stating -the fact, then these days will be adjusted as sick leave (if available). - If an employee takes annual leave from work starting any day of the week, any weekend or -holiday that falls in between will be counted as an annual leave. - During notice period, no annual leave will be allowed even if the leave shows credit balance. -Documentation and Process: - A completed leave application form is to be submitted for approval in advance to the HR -department prior to the anticipated start of the leave accordingly: -Number of leaves Approval time required -1-3 days Before 2 days -4-7 days Before 1 week -More than 7 days Before 1 month - An employee who wishes to obtain annual leaves shall apply in writing to HR in advance before -the commencement of leaves. - If the leave request is refused or postponed on account of company exigencies, the fact of such -refusal or postponement and reasons thereto shall be conveyed to the employee concerned. - If the requested leave is granted the employee will proceed on his leave on the approved date. - If an employee remains absent beyond the approved period of leave, he shall be liable for -disciplinary action unless the employee has justifiable reasons for absenteeism. - Annual leaves may only be availed after prior approval has been granted, however in case of an -emergency, annual leaves may be entertained subject to your supervising authority’s discretion. -Hajj Leave -Definition: -Employees who wish to make their pilgrimage to Mecca during Dhu'l Hijja may do so during -their tenure at MartechSol. These leaves are separate from the employee's annual leaves or any -other leave which he/she is entitled to. - - -Policy: - All full time permanent employees with 4 years of service are entitled for paid Hajj Leave for 10 -days. - Any leaves beyond 10 days needs to come from your annual or casual leaves. - Hajj leave will be granted only once during the whole employment period. - -Documentation and Process: - Your leave application must be submitted three months in advance for the necessary approval. - Copy of your Hajj Passport/Visa needs to be submitted to HR. -Maternity Leave -Definition: -A period of approved absence for female employees that wish to continue employment -at MartechSol after delivery, granted for the purpose of giving birth and taking care of -newborn children. -Policy: - All full-time female employees shall be entitled to 90 days paid maternity leave, taken no later -than two weeks before the expected delivery date and is only applicable after completion of one -year of service. - This entitlement shall be available only twice during the entire period of service. - Premature birth shall be considered as maternity leave. In case of still birth, female employee -shall be entitled to 30 days paid maternity leave. -Documentation and Process: - Maternity Leave form needs to be submitted and approved five months before planned -maternity leave. - -Paternity Leave -Definition: -A period of approved absence for male employees granted after or shortly before the birth of his -child. -Policy: - All full-time male employees are entitled to 3 days paid paternity leave. - This entitlement shall be available only twice during the entire period of service. - - - - Male employees may combine other leaves with paternity leave subject to approval from -supervising authority.  - Premature birth shall be considered as paternity leave. In case of still birth, male employee shall -be entitled to up to 3 days paid paternity leave. -Documentation and Process: - Application for paternity leave should be submitted to HR department at least two months in -advance supported by the gynecologist’s certification. -Bereavement Leave -Definition: -Bereavement leaves are paid leaves for employees that cover absences related to the death of -immediate family members. - Policy: - All full-time employee(s) shall be entitled to 3 days paid bereavement leave. - Bereavement leave will normally be granted unless there are unusual business needs or staffing -requirements. An employee may, with his or her supervisor’s approval, use any available leaves -for additional time off as necessary. - Documentation and Process: - You must apply for bereavement leave immediately after returning from your leave by -submitting your leave application to the HR department after approval from your supervising -authority. -Unauthorized Leaves - Any - absence without prior approval from your reporting authority will be considered as an -unauthorized leave. - All unauthorized and/or unreported absences shall be considered Absences without Leave, -(AWOL). Each unauthorized leave will result in a disciplinary notice and will be added in your file. -3 unauthorized leaves in a quarter will result in a severe disciplinary action. - In case you were not able to inform your reporting authority due to some unavoidable reasons, -you may discuss it with your reporting authority later. Based on discretion, he/she may authorize -your leave. - For employees serving their probationary period, failure to communicate with the respective -authority for more than 3 consecutive days of leave will result in termination. - For permanent employees, failure to communicate with the respective authority for more than 7 -consecutive days of leave will result in termination. - - -Unapproved Absence Without Pay -Definition: -Any leave, in excess of the 10 Casual leaves that you are entitled to, is an Absence without Pay. -These leaves, when taken without prior approval from your supervising authority are considered -Unapproved Absence without Pay. -Policy: - - Unapproved Absence without Pay will result in salary deduction(s) per day absent. - Taking more than two consecutive Absences without Pay prior to the weekend will result in salary -deduction for the weekend. For example, if an employee is absent on Thursday, Friday, and -Saturday then Sunday will also be marked absent. In other words, employees must report to work -on the last working day of the week to avoid further deductions.  -Leave Encashment -Each employee is entitled to a number of Sick, Casual, Annual, and other leaves. However, only Casual -leaves are subject to encashment for specific departments. -Listed below is breakdown of Department eligibility of leave encashment: -Department Casual Annual -Copywriting Yes No -SEO Yes No -Sales Yes Yes -Software Yes No -Networking Yes No -Human Resources Yes No -Admin Yes No -Other Benefits -Loans & Advance Salary -Definition: -The purpose of loans and advance salary is to meet an emergency need that can’t be -accommodated through other financial arrangements. Only genuine emergency needs such as -extraordinary medical costs not covered by insurance, or any other extreme financial or medical -emergency will be considered. The decision to approve or reject a loan/advance salary request will -be taken by a 3-person committee which will comprise of line manager/head of department, a -representative from HR, and a representative from finance. -Policy: - Decisions regarding loan requests will be relayed within 3 to 7 working days. For extremely urgent -matters, the committee will take a decision within 27 hours of request sent. - The decision of the committee will be final and non-negotiable. - A loans/advance salary can only be taken once in 6 months. - - - If an employee has an ongoing loan/advance salary, he/she will not be allowed to avail another -loan/advance salary. - Repayment period is up to 6 months. - Loan amount can’t exceed 3 months’ salary. - Advance salary amount shall not exceed from your monthly salary. -Documentation & Process: - If possible, the loan/advance salary form should be submitted at least a week (7 days) before it is -required. - The form can be acquired from HR and must be completed including all the signatures required. - The necessary documentation & supporting documents should be provided along with the filled -form. -Maternity (Pregnancy) Benefit -Definition: -This benefit provides financial assistance to female and male employees to cover the cost of childbirth and -other delivery related expenses. -Policy: - Male and female employees that have completed at least one year of service can avail maternity -benefit of PKR 50,000. - Maternity benefit can be utilized twice during tenure at MartechSol. - This benefit also covers stillbirth. -Documentation & Process: - Maternity Benefit Form and relevant documentation must be provided at least 60 days before -expected delivery. - Employees will be awarded this benefit a week before date of expected delivery. - In case of premature delivery, employees can either contact Supervising Authority or Admin -Department to arrange the benefit urgently or they may pay out of pocket at the time and avail the -benefit when they rejoin the office. - - -Company Maintained Vehicle -Definition: -MartechSol provides company maintained cars to employees at certain designation levels. The -policy varies across different departments. The company maintained car will be owned and -maintained by the company for the use of the employee. -Policy: - Employees should drive their allotted vehicle in a safe and responsible vehicle. They must abide by -driving laws. - Scheduled maintenance will be carried out by the company after every 5,000 KMs or 3 months -(whichever comes first). - Employees are requested to keep a copy of the following documents in the company vehicle: -o Authority Letter -o Motor Vehicle Tax paper -o Insurance paper -o Copy of the first page of the vehicle’s registration book -Documentation & Process: - Prior to vehicle allocation, employee must provide a copy of valid driving license of self (or driver) -for company record. - All maintenance and wear & tear requests need to be made through submitting a ticket on portal or -by contacting Admin Department. - In case of an accident: -o All accidents must be reported to the Admin Department. -o If the company vehicle is stolen, report the theft immediately to the local police and to the -Admin Department. Obtain a copy of the police report filed. -Healthcare Insurance – For Self & Spouse -Health care benefits are provided to all full-time employees and their spouse. Complete details of this -program are provided below: - “ANNEXURE A” -PROCEDURE FOR UTILIZATION OF HOSPITALIZATION EXPENSE BENEFIT -Emergency Hospitalization -In case of emergency, approach the nearest hospital. In case it is a PANEL HOSPITAL -1. Identify yourself as an NJI insured. -2. Produce Health Card / Letter of Authority -3. Get the treatment -4. Get the treatment on credit, up to the limits available. -5. Pay only the amount that exceeds the entitlement, if any, before discharge. - - -In case it is a NON-PANEL HOSPITAL -1. Inform NJI within 24 hours of the hospitalization. -2. Pay cash for the treatment. -3. Submit all original bills /supporting documents* with our claim form for reimbursement, -within 30 days of discharge from the hospital. -4. Settlement of claim will be done in line of the policy terms. - “ANNEXURE B” -PROCEDURE FOR UTILIZATION OF HOSPITALIZATION EXPENSE BENEFIT -Emergency Hospitalization -If hospitalization is advised by a licensed physician - In case it is a PANEL HOSPITAL -1. Approach any of our panel hospital with our Letter of Authority / Health Card -2. Identify yourself as an NJI insured -3. Get the treatment on credit, up to the limits available. -4. Pay only the amount that exceeds the entitlement, if any, before discharge. -In case it is a NON-PANEL HOSPITAL -1. Send us the cost estimate from the concerned doctor of the treatment with details, for prior -approval. -2. Get the treatment and pay cash for the treatment. -3. Submit all original bills /supporting documents* including our approval letter with our claim -form for reimbursement, within 30 days of discharge from the hospital. -4. Settlement of claim will be done in line of prior approval given and other policy terms. -*Supporting Documents: - Duly completed original NJI Claim Form - Original itemized bill/invoice (breakup of charged) on hospital -bill-book - Discharge card / clinical summary - Copy of NJI card or letter of admission - Diagnostic reports - Doctors prescriptions - Original pharmacy vouchers - Original payment receipts - - -JUBILEE GENERAL INSURANCE COMPANY LTD -GROUP HEALTHCARE INSURANCE PROPOSAL FOR -MartechSol -Hospitalization & Related Benefits -Plan A Plan B Plan C Plan D -H&R Limits (Per Person / Per Year) Rs.500,000 Rs.400,000 Rs.300,000 Rs.150,000 -Room & Board (per day) Rs.16,360 Rs.6,720 Rs.6,720 Rs.1,800 -Per Hospitalization -Pre-Hospitalization Sub Limit (Diagnosis, 30 Days 30 Days 30 Days 30 Days -Consultation, & Medicines) -Post-Hospitalization Sub Limit (Follow-Ups) 30 Days 30 Days 30 Days 30 Days -Daycare Surgeries & Specialized -Investigations In Outpatient Settings -Including but not limited to: -COVERED -Dialysis, Cataract Surgery, MRI, CT Scan, -Endoscopy, Thallium Scan, Angiography, Treatment -of Fractures, Local Road Ambulance for -Emergencies only, Emergency Dental Treatment -due to accidental injuries within 48 hours (for pain -relief only). - Plan A Presidential Layer - Plan B Associate Managers, Managers & Senior Managers - Plan C Executives, Senior Executives & Assistant Managers - Plan D Office Boys & Other Support Staff - Send us the cost estimate from the concerned doctor of the treatment with details, for prior -approval. Get the treatment and pay cash for the treatment. - Submit all original bills /supporting documents* including our approval letter with our claim form -for reimbursement, within 30 days of discharge from the hospital. - Settlement of claim will be done in line of prior approval given and other policy terms. -Supporting documents -Itemized hospital bill -Discharge card/clinical summary -Diagnostic reports -Prescriptions -Payment receipts - - -EOBI -Employee Old Age Benefit Income is a benefit provided to all full-time employees upon confirmation. -The sole purpose of EOBI is to provide compulsory social insurance to employees. It extends following -benefits to insured persons or their survivors: -• Old-Age Pension -• Survivor's Pension -• Invalidity Pension -• Old-Age Grant  -RATES: -• 1- Employer's Contribution @ 5 % of the worker's minimum wages (i.e. Rs. 8000) - Rs. 400/= Per -Month -• 2- Employee's Contribution @ 1 % of the worker's minimum wages (i.e. Rs.8000) - Rs. 80/= Per -Month -Provident Fund -Provident Fund is an investment/ saving fund contributed to - by employees & employers, out of which a -lump sum amount is provided to each employee on retirement. This benefit is provided to all full time, -confirmed employees. -Click Here To View PF Policies in Detail - - diff --git a/backup_2026-05-04/floaitng-icon/KhloeSAC.lottie b/backup_2026-05-04/floaitng-icon/KhloeSAC.lottie deleted file mode 100644 index dd4649d7db83bd2603fe9137d6e804d465753102..0000000000000000000000000000000000000000 --- a/backup_2026-05-04/floaitng-icon/KhloeSAC.lottie +++ /dev/null @@ -1,3 +0,0 @@ -version https://git-lfs.github.com/spec/v1 -oid sha256:532a144eeea5a1a6e49ab22ae4c0dc4f94e12cf8540e3514f1bc676d929eeac2 -size 462684 diff --git a/backup_2026-05-04/get_models.py b/backup_2026-05-04/get_models.py deleted file mode 100644 index e7bcf221c211cef860ae9402c8e3adbaa648b025..0000000000000000000000000000000000000000 --- a/backup_2026-05-04/get_models.py +++ /dev/null @@ -1,25 +0,0 @@ -import asyncio -import os -import httpx -from dotenv import load_dotenv - -load_dotenv() - -async def main(): - api_key = os.getenv("GROQ_API_KEY") - if not api_key: - print("No API Key") - return - url = "https://api.groq.com/openai/v1/models" - headers = {"Authorization": f"Bearer {api_key}"} - async with httpx.AsyncClient() as client: - resp = await client.get(url, headers=headers) - if resp.status_code == 200: - models = resp.json().get("data", []) - for m in models: - if "deepseek" in m["id"].lower() or "qwen" in m["id"].lower(): - print(m["id"]) - else: - print(resp.status_code, resp.text) - -asyncio.run(main()) diff --git a/backup_2026-05-04/martech_sol_logo.jpg b/backup_2026-05-04/martech_sol_logo.jpg deleted file mode 100644 index 4df90316265efa116ebc6a35a7a300c8515efdf1..0000000000000000000000000000000000000000 Binary files a/backup_2026-05-04/martech_sol_logo.jpg and /dev/null differ diff --git a/backup_2026-05-04/requirements.txt b/backup_2026-05-04/requirements.txt deleted file mode 100644 index 88a16cc36cae5251255217fcc5504d57b676f9c3..0000000000000000000000000000000000000000 --- a/backup_2026-05-04/requirements.txt +++ /dev/null @@ -1,14 +0,0 @@ -fastapi -uvicorn[standard] -faiss-cpu -sentence-transformers -pydantic-settings -pypdf -python-dotenv -numpy -httpx -gradio>=5.0.0 -python-multipart -pandas -openpyxl -rank_bm25 diff --git a/backup_2026-05-04/static/addon.html b/backup_2026-05-04/static/addon.html deleted file mode 100644 index 7d1cf1c953f2eef06ab8a7805aa69c18b79b3442..0000000000000000000000000000000000000000 --- a/backup_2026-05-04/static/addon.html +++ /dev/null @@ -1,543 +0,0 @@ - - - - - -Martechsol Assistant Widget - - - - - - - -
-
- -
- - -
- - -
- -
-
-
- Martechsol Assistant -
- - -
-
Welcome! How can I help you today?
-
- -
- - -
-
-
- - - - - \ No newline at end of file diff --git a/backup_2026-05-04/static/bot-icon.lottie b/backup_2026-05-04/static/bot-icon.lottie deleted file mode 100644 index dd4649d7db83bd2603fe9137d6e804d465753102..0000000000000000000000000000000000000000 --- a/backup_2026-05-04/static/bot-icon.lottie +++ /dev/null @@ -1,3 +0,0 @@ -version https://git-lfs.github.com/spec/v1 -oid sha256:532a144eeea5a1a6e49ab22ae4c0dc4f94e12cf8540e3514f1bc676d929eeac2 -size 462684 diff --git a/backup_2026-05-04/static/chat-loader.js b/backup_2026-05-04/static/chat-loader.js deleted file mode 100644 index 351528c2cf89ce5f4464b3ae88268f70473dd2f8..0000000000000000000000000000000000000000 --- a/backup_2026-05-04/static/chat-loader.js +++ /dev/null @@ -1,48 +0,0 @@ -/** - * Martechsol Assistant Loader - * Usage: - */ -(function() { - const baseUrl = 'https://joedown11-chatrag.hf.space'; - - // Create container - const container = document.createElement('div'); - container.id = 'martech-chat-widget-container'; - document.body.appendChild(container); - - // Fetch the addon HTML from the dedicated /widget endpoint - fetch(`${baseUrl}/widget?v=${Date.now()}`) // Cache busting - .then(response => response.text()) - .then(html => { - // Create a temporary element to parse the HTML - const parser = new DOMParser(); - const doc = parser.parseFromString(html, 'text/html'); - - // 1. Extract and inject styles - doc.querySelectorAll('style').forEach(st => { - document.head.appendChild(st.cloneNode(true)); - }); - - // 2. Extract and inject body content - const wrapper = doc.querySelector('#martech-chat-wrapper'); - if (wrapper) { - container.innerHTML = wrapper.outerHTML; - } else { - // Fallback: just take the body content - container.innerHTML = doc.body.innerHTML; - } - - // 3. Extract and execute scripts (critical for logic) - doc.querySelectorAll('script').forEach(oldScript => { - const newScript = document.createElement('script'); - Array.from(oldScript.attributes).forEach(attr => { - newScript.setAttribute(attr.name, attr.value); - }); - newScript.innerHTML = oldScript.innerHTML; - document.body.appendChild(newScript); - }); - - console.log("Martechsol Assistant Loaded Successfully"); - }) - .catch(err => console.error("Failed to load Martechsol Assistant:", err)); -})(); diff --git a/backup_2026-05-04/test_groq.py b/backup_2026-05-04/test_groq.py deleted file mode 100644 index 881464dbb8b59db2da7b164fa708b46835d7b2e8..0000000000000000000000000000000000000000 --- a/backup_2026-05-04/test_groq.py +++ /dev/null @@ -1,26 +0,0 @@ -import asyncio -import os -import httpx -from dotenv import load_dotenv - -load_dotenv() - -async def main(): - api_key = os.getenv("GROQ_API_KEY") - url = "https://api.groq.com/openai/v1/chat/completions" - headers = { - "Authorization": f"Bearer {api_key}", - "Content-Type": "application/json" - } - payload = { - "model": "deepseek-r1-distill-llama-70b", - "temperature": 0.0, - "max_tokens": 1200, - "messages": [{"role": "system", "content": "hello"}, {"role": "user", "content": "test"}] - } - async with httpx.AsyncClient() as client: - resp = await client.post(url, headers=headers, json=payload) - print(resp.status_code) - print(resp.text) - -asyncio.run(main()) diff --git a/backup_2026-05-04/wordpress code/WP_INTEGRATION_SCRIPT.txt b/backup_2026-05-04/wordpress code/WP_INTEGRATION_SCRIPT.txt deleted file mode 100644 index f4d892fb2e96f15d663b36c219dcc345b9232dd8..0000000000000000000000000000000000000000 --- a/backup_2026-05-04/wordpress code/WP_INTEGRATION_SCRIPT.txt +++ /dev/null @@ -1,12 +0,0 @@ - - - diff --git a/backup_2026-05-04/wordpress code/addon.html b/backup_2026-05-04/wordpress code/addon.html deleted file mode 100644 index aefe151d3d86fc85a597e5dcb1fe84c4f42d6c77..0000000000000000000000000000000000000000 --- a/backup_2026-05-04/wordpress code/addon.html +++ /dev/null @@ -1,477 +0,0 @@ - - - - - -Martechsol Assistant Widget - - - - - - - -
-
- -
- - -
- - -
- -
-
-
- Martechsol Assistant -
- - -
-
Welcome! How can I help you today?
-
- -
- - -
-
-
- - - - - \ No newline at end of file diff --git a/backup_2026-05-04/wordpress code/addon_backup.html b/backup_2026-05-04/wordpress code/addon_backup.html deleted file mode 100644 index d1155a0d2593c40e1002009b2da9fc0b8ea7cb59..0000000000000000000000000000000000000000 --- a/backup_2026-05-04/wordpress code/addon_backup.html +++ /dev/null @@ -1,175 +0,0 @@ - - - - - -
-
- -
- - -
- - -
- -
-
-
- Martechsol Assistant -
- -
-
- - \ No newline at end of file diff --git a/backup_2026-05-13_pre_intelligent_retrieval/.env.example b/backup_2026-05-13_pre_intelligent_retrieval/.env.example deleted file mode 100644 index 87dd71c6445a387445aac1d442e53156372719cd..0000000000000000000000000000000000000000 --- a/backup_2026-05-13_pre_intelligent_retrieval/.env.example +++ /dev/null @@ -1,19 +0,0 @@ -LLM_PROVIDER=fireworks -FIREWORKS_API_KEY=your_fireworks_api_key -FIREWORKS_MODEL=accounts/fireworks/models/qwen3-32b -FIREWORKS_REWRITE_MODEL=accounts/fireworks/models/llama-v3p1-8b-instruct -GROQ_API_KEY=your_groq_api_key -GROQ_MODEL=qwen/qwen3-32b -GROQ_REWRITE_MODEL=llama-3.1-8b-instant -HF_API_KEY=your_huggingface_api_key -HF_MODEL=meta-llama/Llama-3.1-8B-Instruct -EMBEDDING_MODEL=BAAI/bge-small-en-v1.5 -DOCS_DIR=docs -INDEX_DIR=data/index -SESSIONS_DIR=data/sessions -TOP_K=4 -CORS_ALLOW_ORIGINS=* -API_KEY= -RATE_LIMIT_REQUESTS=60 -RATE_LIMIT_WINDOW_SECONDS=60 -RAG_API_URL= diff --git a/backup_2026-05-13_pre_intelligent_retrieval/.gitattributes b/backup_2026-05-13_pre_intelligent_retrieval/.gitattributes deleted file mode 100644 index f18a35d12ad3bd5aff5f573ece54f1bd056249aa..0000000000000000000000000000000000000000 --- a/backup_2026-05-13_pre_intelligent_retrieval/.gitattributes +++ /dev/null @@ -1,2 +0,0 @@ -*.lottie filter=lfs diff=lfs merge=lfs -text -*.pdf filter=lfs diff=lfs merge=lfs -text diff --git a/backup_2026-05-13_pre_intelligent_retrieval/.gitignore b/backup_2026-05-13_pre_intelligent_retrieval/.gitignore deleted file mode 100644 index 6f781509d5f4c292645a2425e9d6291a5aa33b41..0000000000000000000000000000000000000000 --- a/backup_2026-05-13_pre_intelligent_retrieval/.gitignore +++ /dev/null @@ -1,9 +0,0 @@ -.env -__pycache__/ -.pytest_cache/ -.mypy_cache/ -.venv/ -venv/ -# Ignoring data directory to prevent pushing binary index files -data/index/ -data/sessions/ diff --git a/backup_2026-05-13_pre_intelligent_retrieval/CONFIG_NOTES.md b/backup_2026-05-13_pre_intelligent_retrieval/CONFIG_NOTES.md deleted file mode 100644 index 5517ac20537f1769b1bbc6d630b79b85e67b712f..0000000000000000000000000000000000000000 --- a/backup_2026-05-13_pre_intelligent_retrieval/CONFIG_NOTES.md +++ /dev/null @@ -1,38 +0,0 @@ -# Configuration Backup - Martechsol RAG Chatbot -**Date:** 2026-04-28 -**Source:** C:\Users\DELL\Desktop\RAG_backup2 -**Destination:** D:\RAG_Backup_2026_04_28 - -## Core Settings (from app/core/config.py) -- **App Name:** Fast RAG Chatbot -- **LLM Provider:** groq (default) -- **Groq Model:** llama-3.1-8b-instant -- **HF Model:** meta-llama/Llama-3.1-8B-Instruct -- **Embedding Model:** BAAI/bge-small-en-v1.5 -- **Docs Directory:** docs/ -- **Index Directory:** data/index/ -- **Sessions Directory:** data/sessions/ -- **Chunk Size:** 420 tokens -- **Overlap:** 80 tokens -- **Top K:** 4 - -## Admin Credentials -- **Username:** martech_admin -- **Password:** martech_admin_303 -- **OTP Expiry:** 300 seconds (5 minutes) - -## Security -- **Auth Method:** HTTP Basic Auth + OTP (email-based) -- **Email for OTP:** randomjoedown@gmail.com -- **SMTP Server:** smtp.gmail.com (Port 465) - -## API / Integration -- **Endpoint:** /api/chat -- **Widget Endpoint:** /widget -- **Admin Endpoint:** /admin -- **CORS:** Allowed for all (*) for testing - -## Infrastructure -- **Hugging Face Path:** /data (for persistent storage) -- **Local Path:** data/ (for local storage) -- **Dockerfile:** Based on python:3.10-slim, serves on port 7860 diff --git a/backup_2026-05-13_pre_intelligent_retrieval/CUSTOMIZATION_GUIDE.md b/backup_2026-05-13_pre_intelligent_retrieval/CUSTOMIZATION_GUIDE.md deleted file mode 100644 index 22d70e37fd9c17d9a66426cec7158d449be17064..0000000000000000000000000000000000000000 --- a/backup_2026-05-13_pre_intelligent_retrieval/CUSTOMIZATION_GUIDE.md +++ /dev/null @@ -1,55 +0,0 @@ -# Martechsol Assistant Customization Guide - -This guide explains how to customize the visual appearance and behavior of your Gradio-based chat interface. - -## 1. Modifying the UI Layout (app/ui_gradio.py) - -The interface is built using `gradio.Blocks`. You can modify the structure in the `with gr.Blocks(...) as demo:` section. - -### Common Components: -- **`chatbot`**: The main chat window. We use `elem_id="chatbot-window"` for CSS targeting. -- **`msg`**: The text input box. -- **`send`**: The primary action button. - -## 2. Custom CSS (app/ui_gradio.py) - -At the top of `app/ui_gradio.py`, there is a `custom_css` string. This is the most powerful way to change the look of your app. - -### How to target elements: -We use `elem_id` in Python to give elements stable names. - -Example: -```python -chatbot = gr.Chatbot(elem_id="my-custom-chat") -``` -Then in CSS: -```css -#my-custom-chat { - background-color: #f0f0f0; -} -``` - -### Important Selectors: -- **`footer`**: Targets the Gradio branding at the bottom. -- **`.gradio-container`**: The main outer wrapper of your app. -- **`[data-testid="user"]`**: Targets user messages. -- **`[data-testid="bot"]`**: Targets assistant messages. - -## 3. Persistent Data on Hugging Face - -Your data is stored in the `/data` volume. -- **Sessions**: `/data/sessions/` -- **FAISS Index**: `/data/index/` - -To manage these, you can use the **Admin Panel** at `/admin`. - -## 4. Tips for Widget Embedding - -If you are embedding this as an iframe (like in `addon.html`): -1. **Remove redundant headers**: We hide the Markdown titles in the CSS (`#title-area { display: none; }`) so the iframe looks like a part of your page. -2. **Handle height**: We set `height: auto !important` and `min-height: 40px !important` on the chatbot so it doesn't take up 500px by default. -3. **Transparent Background**: You can add `body { background-color: transparent !important; }` to the `custom_css` if your theme supports it. - -## 5. Deployment - -Whenever you save a file, the changes are automatically pushed to your Hugging Face Space. The Space will rebuild automatically (takes about 1-2 minutes). diff --git a/backup_2026-05-13_pre_intelligent_retrieval/Dockerfile b/backup_2026-05-13_pre_intelligent_retrieval/Dockerfile deleted file mode 100644 index b2137aa0e63b826e988226e4902e1d03496b7078..0000000000000000000000000000000000000000 --- a/backup_2026-05-13_pre_intelligent_retrieval/Dockerfile +++ /dev/null @@ -1,31 +0,0 @@ -FROM python:3.11-slim - -# Set environment variables -ENV PYTHONDONTWRITEBYTECODE=1 -ENV PYTHONUNBUFFERED=1 -ENV HOME=/home/user - -# Create a non-root user for Hugging Face -RUN useradd -m -u 1000 user -WORKDIR $HOME/app - -# Pre-create data directories with correct ownership -RUN mkdir -p $HOME/app/data/index $HOME/app/data/sessions && \ - chown -R user:user $HOME/app - -# Switch to non-root user -USER user -ENV PATH=$HOME/.local/bin:$PATH - -# Install dependencies -COPY --chown=user requirements.txt . -RUN pip install --no-cache-dir --user -r requirements.txt - -# Copy application code -COPY --chown=user . . - -# Expose port -EXPOSE 7860 - -# Start application -CMD ["python", "-m", "uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "7860", "--workers", "1"] diff --git a/backup_2026-05-13_pre_intelligent_retrieval/PHPMailer/Exception.php b/backup_2026-05-13_pre_intelligent_retrieval/PHPMailer/Exception.php deleted file mode 100644 index 09c1a2cfefaa986c846b41743e856c945beff3cd..0000000000000000000000000000000000000000 --- a/backup_2026-05-13_pre_intelligent_retrieval/PHPMailer/Exception.php +++ /dev/null @@ -1,40 +0,0 @@ - - * @author Jim Jagielski (jimjag) - * @author Andy Prevost (codeworxtech) - * @author Brent R. Matzelle (original founder) - * @copyright 2012 - 2020 Marcus Bointon - * @copyright 2010 - 2012 Jim Jagielski - * @copyright 2004 - 2009 Andy Prevost - * @license https://www.gnu.org/licenses/old-licenses/lgpl-2.1.html GNU Lesser General Public License - * @note This program is distributed in the hope that it will be useful - WITHOUT - * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or - * FITNESS FOR A PARTICULAR PURPOSE. - */ - -namespace PHPMailer\PHPMailer; - -/** - * PHPMailer exception handler. - * - * @author Marcus Bointon - */ -class Exception extends \Exception -{ - /** - * Prettify error message output. - * - * @return string - */ - public function errorMessage() - { - return '' . htmlspecialchars($this->getMessage(), ENT_COMPAT | ENT_HTML401) . "
\n"; - } -} diff --git a/backup_2026-05-13_pre_intelligent_retrieval/PHPMailer/PHPMailer.php b/backup_2026-05-13_pre_intelligent_retrieval/PHPMailer/PHPMailer.php deleted file mode 100644 index 31731594f12ec7bb6bd2eca0a990c91233f2137a..0000000000000000000000000000000000000000 --- a/backup_2026-05-13_pre_intelligent_retrieval/PHPMailer/PHPMailer.php +++ /dev/null @@ -1,5250 +0,0 @@ - - * @author Jim Jagielski (jimjag) - * @author Andy Prevost (codeworxtech) - * @author Brent R. Matzelle (original founder) - * @copyright 2012 - 2020 Marcus Bointon - * @copyright 2010 - 2012 Jim Jagielski - * @copyright 2004 - 2009 Andy Prevost - * @license https://www.gnu.org/licenses/old-licenses/lgpl-2.1.html GNU Lesser General Public License - * @note This program is distributed in the hope that it will be useful - WITHOUT - * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or - * FITNESS FOR A PARTICULAR PURPOSE. - */ - -namespace PHPMailer\PHPMailer; - -/** - * PHPMailer - PHP email creation and transport class. - * - * @author Marcus Bointon (Synchro/coolbru) - * @author Jim Jagielski (jimjag) - * @author Andy Prevost (codeworxtech) - * @author Brent R. Matzelle (original founder) - */ -class PHPMailer -{ - const CHARSET_ASCII = 'us-ascii'; - const CHARSET_ISO88591 = 'iso-8859-1'; - const CHARSET_UTF8 = 'utf-8'; - - const CONTENT_TYPE_PLAINTEXT = 'text/plain'; - const CONTENT_TYPE_TEXT_CALENDAR = 'text/calendar'; - const CONTENT_TYPE_TEXT_HTML = 'text/html'; - const CONTENT_TYPE_MULTIPART_ALTERNATIVE = 'multipart/alternative'; - const CONTENT_TYPE_MULTIPART_MIXED = 'multipart/mixed'; - const CONTENT_TYPE_MULTIPART_RELATED = 'multipart/related'; - - const ENCODING_7BIT = '7bit'; - const ENCODING_8BIT = '8bit'; - const ENCODING_BASE64 = 'base64'; - const ENCODING_BINARY = 'binary'; - const ENCODING_QUOTED_PRINTABLE = 'quoted-printable'; - - const ENCRYPTION_STARTTLS = 'tls'; - const ENCRYPTION_SMTPS = 'ssl'; - - const ICAL_METHOD_REQUEST = 'REQUEST'; - const ICAL_METHOD_PUBLISH = 'PUBLISH'; - const ICAL_METHOD_REPLY = 'REPLY'; - const ICAL_METHOD_ADD = 'ADD'; - const ICAL_METHOD_CANCEL = 'CANCEL'; - const ICAL_METHOD_REFRESH = 'REFRESH'; - const ICAL_METHOD_COUNTER = 'COUNTER'; - const ICAL_METHOD_DECLINECOUNTER = 'DECLINECOUNTER'; - - /** - * Email priority. - * Options: null (default), 1 = High, 3 = Normal, 5 = low. - * When null, the header is not set at all. - * - * @var int|null - */ - public $Priority; - - /** - * The character set of the message. - * - * @var string - */ - public $CharSet = self::CHARSET_ISO88591; - - /** - * The MIME Content-type of the message. - * - * @var string - */ - public $ContentType = self::CONTENT_TYPE_PLAINTEXT; - - /** - * The message encoding. - * Options: "8bit", "7bit", "binary", "base64", and "quoted-printable". - * - * @var string - */ - public $Encoding = self::ENCODING_8BIT; - - /** - * Holds the most recent mailer error message. - * - * @var string - */ - public $ErrorInfo = ''; - - /** - * The From email address for the message. - * - * @var string - */ - public $From = ''; - - /** - * The From name of the message. - * - * @var string - */ - public $FromName = ''; - - /** - * The envelope sender of the message. - * This will usually be turned into a Return-Path header by the receiver, - * and is the address that bounces will be sent to. - * If not empty, will be passed via `-f` to sendmail or as the 'MAIL FROM' value over SMTP. - * - * @var string - */ - public $Sender = ''; - - /** - * The Subject of the message. - * - * @var string - */ - public $Subject = ''; - - /** - * An HTML or plain text message body. - * If HTML then call isHTML(true). - * - * @var string - */ - public $Body = ''; - - /** - * The plain-text message body. - * This body can be read by mail clients that do not have HTML email - * capability such as mutt & Eudora. - * Clients that can read HTML will view the normal Body. - * - * @var string - */ - public $AltBody = ''; - - /** - * An iCal message part body. - * Only supported in simple alt or alt_inline message types - * To generate iCal event structures, use classes like EasyPeasyICS or iCalcreator. - * - * @see https://kigkonsult.se/iCalcreator/ - * - * @var string - */ - public $Ical = ''; - - /** - * Value-array of "method" in Contenttype header "text/calendar" - * - * @var string[] - */ - protected static $IcalMethods = [ - self::ICAL_METHOD_REQUEST, - self::ICAL_METHOD_PUBLISH, - self::ICAL_METHOD_REPLY, - self::ICAL_METHOD_ADD, - self::ICAL_METHOD_CANCEL, - self::ICAL_METHOD_REFRESH, - self::ICAL_METHOD_COUNTER, - self::ICAL_METHOD_DECLINECOUNTER, - ]; - - /** - * The complete compiled MIME message body. - * - * @var string - */ - protected $MIMEBody = ''; - - /** - * The complete compiled MIME message headers. - * - * @var string - */ - protected $MIMEHeader = ''; - - /** - * Extra headers that createHeader() doesn't fold in. - * - * @var string - */ - protected $mailHeader = ''; - - /** - * Word-wrap the message body to this number of chars. - * Set to 0 to not wrap. A useful value here is 78, for RFC2822 section 2.1.1 compliance. - * - * @see static::STD_LINE_LENGTH - * - * @var int - */ - public $WordWrap = 0; - - /** - * Which method to use to send mail. - * Options: "mail", "sendmail", or "smtp". - * - * @var string - */ - public $Mailer = 'mail'; - - /** - * The path to the sendmail program. - * - * @var string - */ - public $Sendmail = '/usr/sbin/sendmail'; - - /** - * Whether mail() uses a fully sendmail-compatible MTA. - * One which supports sendmail's "-oi -f" options. - * - * @var bool - */ - public $UseSendmailOptions = true; - - /** - * The email address that a reading confirmation should be sent to, also known as read receipt. - * - * @var string - */ - public $ConfirmReadingTo = ''; - - /** - * The hostname to use in the Message-ID header and as default HELO string. - * If empty, PHPMailer attempts to find one with, in order, - * $_SERVER['SERVER_NAME'], gethostname(), php_uname('n'), or the value - * 'localhost.localdomain'. - * - * @see PHPMailer::$Helo - * - * @var string - */ - public $Hostname = ''; - - /** - * An ID to be used in the Message-ID header. - * If empty, a unique id will be generated. - * You can set your own, but it must be in the format "", - * as defined in RFC5322 section 3.6.4 or it will be ignored. - * - * @see https://www.rfc-editor.org/rfc/rfc5322#section-3.6.4 - * - * @var string - */ - public $MessageID = ''; - - /** - * The message Date to be used in the Date header. - * If empty, the current date will be added. - * - * @var string - */ - public $MessageDate = ''; - - /** - * SMTP hosts. - * Either a single hostname or multiple semicolon-delimited hostnames. - * You can also specify a different port - * for each host by using this format: [hostname:port] - * (e.g. "smtp1.example.com:25;smtp2.example.com"). - * You can also specify encryption type, for example: - * (e.g. "tls://smtp1.example.com:587;ssl://smtp2.example.com:465"). - * Hosts will be tried in order. - * - * @var string - */ - public $Host = 'localhost'; - - /** - * The default SMTP server port. - * - * @var int - */ - public $Port = 25; - - /** - * The SMTP HELO/EHLO name used for the SMTP connection. - * Default is $Hostname. If $Hostname is empty, PHPMailer attempts to find - * one with the same method described above for $Hostname. - * - * @see PHPMailer::$Hostname - * - * @var string - */ - public $Helo = ''; - - /** - * What kind of encryption to use on the SMTP connection. - * Options: '', static::ENCRYPTION_STARTTLS, or static::ENCRYPTION_SMTPS. - * - * @var string - */ - public $SMTPSecure = ''; - - /** - * Whether to enable TLS encryption automatically if a server supports it, - * even if `SMTPSecure` is not set to 'tls'. - * Be aware that in PHP >= 5.6 this requires that the server's certificates are valid. - * - * @var bool - */ - public $SMTPAutoTLS = true; - - /** - * Whether to use SMTP authentication. - * Uses the Username and Password properties. - * - * @see PHPMailer::$Username - * @see PHPMailer::$Password - * - * @var bool - */ - public $SMTPAuth = false; - - /** - * Options array passed to stream_context_create when connecting via SMTP. - * - * @var array - */ - public $SMTPOptions = []; - - /** - * SMTP username. - * - * @var string - */ - public $Username = ''; - - /** - * SMTP password. - * - * @var string - */ - public $Password = ''; - - /** - * SMTP authentication type. Options are CRAM-MD5, LOGIN, PLAIN, XOAUTH2. - * If not specified, the first one from that list that the server supports will be selected. - * - * @var string - */ - public $AuthType = ''; - - /** - * SMTP SMTPXClient command attributes - * - * @var array - */ - protected $SMTPXClient = []; - - /** - * An implementation of the PHPMailer OAuthTokenProvider interface. - * - * @var OAuthTokenProvider - */ - protected $oauth; - - /** - * The SMTP server timeout in seconds. - * Default of 5 minutes (300sec) is from RFC2821 section 4.5.3.2. - * - * @var int - */ - public $Timeout = 300; - - /** - * Comma separated list of DSN notifications - * 'NEVER' under no circumstances a DSN must be returned to the sender. - * If you use NEVER all other notifications will be ignored. - * 'SUCCESS' will notify you when your mail has arrived at its destination. - * 'FAILURE' will arrive if an error occurred during delivery. - * 'DELAY' will notify you if there is an unusual delay in delivery, but the actual - * delivery's outcome (success or failure) is not yet decided. - * - * @see https://www.rfc-editor.org/rfc/rfc3461.html#section-4.1 for more information about NOTIFY - */ - public $dsn = ''; - - /** - * SMTP class debug output mode. - * Debug output level. - * Options: - * @see SMTP::DEBUG_OFF: No output - * @see SMTP::DEBUG_CLIENT: Client messages - * @see SMTP::DEBUG_SERVER: Client and server messages - * @see SMTP::DEBUG_CONNECTION: As SERVER plus connection status - * @see SMTP::DEBUG_LOWLEVEL: Noisy, low-level data output, rarely needed - * - * @see SMTP::$do_debug - * - * @var int - */ - public $SMTPDebug = 0; - - /** - * How to handle debug output. - * Options: - * * `echo` Output plain-text as-is, appropriate for CLI - * * `html` Output escaped, line breaks converted to `
`, appropriate for browser output - * * `error_log` Output to error log as configured in php.ini - * By default PHPMailer will use `echo` if run from a `cli` or `cli-server` SAPI, `html` otherwise. - * Alternatively, you can provide a callable expecting two params: a message string and the debug level: - * - * ```php - * $mail->Debugoutput = function($str, $level) {echo "debug level $level; message: $str";}; - * ``` - * - * Alternatively, you can pass in an instance of a PSR-3 compatible logger, though only `debug` - * level output is used: - * - * ```php - * $mail->Debugoutput = new myPsr3Logger; - * ``` - * - * @see SMTP::$Debugoutput - * - * @var string|callable|\Psr\Log\LoggerInterface - */ - public $Debugoutput = 'echo'; - - /** - * Whether to keep the SMTP connection open after each message. - * If this is set to true then the connection will remain open after a send, - * and closing the connection will require an explicit call to smtpClose(). - * It's a good idea to use this if you are sending multiple messages as it reduces overhead. - * See the mailing list example for how to use it. - * - * @var bool - */ - public $SMTPKeepAlive = false; - - /** - * Whether to split multiple to addresses into multiple messages - * or send them all in one message. - * Only supported in `mail` and `sendmail` transports, not in SMTP. - * - * @var bool - * - * @deprecated 6.0.0 PHPMailer isn't a mailing list manager! - */ - public $SingleTo = false; - - /** - * Storage for addresses when SingleTo is enabled. - * - * @var array - */ - protected $SingleToArray = []; - - /** - * Whether to generate VERP addresses on send. - * Only applicable when sending via SMTP. - * - * @see https://en.wikipedia.org/wiki/Variable_envelope_return_path - * @see https://www.postfix.org/VERP_README.html Postfix VERP info - * - * @var bool - */ - public $do_verp = false; - - /** - * Whether to allow sending messages with an empty body. - * - * @var bool - */ - public $AllowEmpty = false; - - /** - * DKIM selector. - * - * @var string - */ - public $DKIM_selector = ''; - - /** - * DKIM Identity. - * Usually the email address used as the source of the email. - * - * @var string - */ - public $DKIM_identity = ''; - - /** - * DKIM passphrase. - * Used if your key is encrypted. - * - * @var string - */ - public $DKIM_passphrase = ''; - - /** - * DKIM signing domain name. - * - * @example 'example.com' - * - * @var string - */ - public $DKIM_domain = ''; - - /** - * DKIM Copy header field values for diagnostic use. - * - * @var bool - */ - public $DKIM_copyHeaderFields = true; - - /** - * DKIM Extra signing headers. - * - * @example ['List-Unsubscribe', 'List-Help'] - * - * @var array - */ - public $DKIM_extraHeaders = []; - - /** - * DKIM private key file path. - * - * @var string - */ - public $DKIM_private = ''; - - /** - * DKIM private key string. - * - * If set, takes precedence over `$DKIM_private`. - * - * @var string - */ - public $DKIM_private_string = ''; - - /** - * Callback Action function name. - * - * The function that handles the result of the send email action. - * It is called out by send() for each email sent. - * - * Value can be any php callable: https://www.php.net/is_callable - * - * Parameters: - * bool $result result of the send action - * array $to email addresses of the recipients - * array $cc cc email addresses - * array $bcc bcc email addresses - * string $subject the subject - * string $body the email body - * string $from email address of sender - * string $extra extra information of possible use - * "smtp_transaction_id' => last smtp transaction id - * - * @var string - */ - public $action_function = ''; - - /** - * What to put in the X-Mailer header. - * Options: An empty string for PHPMailer default, whitespace/null for none, or a string to use. - * - * @var string|null - */ - public $XMailer = ''; - - /** - * Which validator to use by default when validating email addresses. - * May be a callable to inject your own validator, but there are several built-in validators. - * The default validator uses PHP's FILTER_VALIDATE_EMAIL filter_var option. - * - * @see PHPMailer::validateAddress() - * - * @var string|callable - */ - public static $validator = 'php'; - - /** - * An instance of the SMTP sender class. - * - * @var SMTP - */ - protected $smtp; - - /** - * The array of 'to' names and addresses. - * - * @var array - */ - protected $to = []; - - /** - * The array of 'cc' names and addresses. - * - * @var array - */ - protected $cc = []; - - /** - * The array of 'bcc' names and addresses. - * - * @var array - */ - protected $bcc = []; - - /** - * The array of reply-to names and addresses. - * - * @var array - */ - protected $ReplyTo = []; - - /** - * An array of all kinds of addresses. - * Includes all of $to, $cc, $bcc. - * - * @see PHPMailer::$to - * @see PHPMailer::$cc - * @see PHPMailer::$bcc - * - * @var array - */ - protected $all_recipients = []; - - /** - * An array of names and addresses queued for validation. - * In send(), valid and non duplicate entries are moved to $all_recipients - * and one of $to, $cc, or $bcc. - * This array is used only for addresses with IDN. - * - * @see PHPMailer::$to - * @see PHPMailer::$cc - * @see PHPMailer::$bcc - * @see PHPMailer::$all_recipients - * - * @var array - */ - protected $RecipientsQueue = []; - - /** - * An array of reply-to names and addresses queued for validation. - * In send(), valid and non duplicate entries are moved to $ReplyTo. - * This array is used only for addresses with IDN. - * - * @see PHPMailer::$ReplyTo - * - * @var array - */ - protected $ReplyToQueue = []; - - /** - * The array of attachments. - * - * @var array - */ - protected $attachment = []; - - /** - * The array of custom headers. - * - * @var array - */ - protected $CustomHeader = []; - - /** - * The most recent Message-ID (including angular brackets). - * - * @var string - */ - protected $lastMessageID = ''; - - /** - * The message's MIME type. - * - * @var string - */ - protected $message_type = ''; - - /** - * The array of MIME boundary strings. - * - * @var array - */ - protected $boundary = []; - - /** - * The array of available text strings for the current language. - * - * @var array - */ - protected $language = []; - - /** - * The number of errors encountered. - * - * @var int - */ - protected $error_count = 0; - - /** - * The S/MIME certificate file path. - * - * @var string - */ - protected $sign_cert_file = ''; - - /** - * The S/MIME key file path. - * - * @var string - */ - protected $sign_key_file = ''; - - /** - * The optional S/MIME extra certificates ("CA Chain") file path. - * - * @var string - */ - protected $sign_extracerts_file = ''; - - /** - * The S/MIME password for the key. - * Used only if the key is encrypted. - * - * @var string - */ - protected $sign_key_pass = ''; - - /** - * Whether to throw exceptions for errors. - * - * @var bool - */ - protected $exceptions = false; - - /** - * Unique ID used for message ID and boundaries. - * - * @var string - */ - protected $uniqueid = ''; - - /** - * The PHPMailer Version number. - * - * @var string - */ - const VERSION = '6.9.3'; - - /** - * Error severity: message only, continue processing. - * - * @var int - */ - const STOP_MESSAGE = 0; - - /** - * Error severity: message, likely ok to continue processing. - * - * @var int - */ - const STOP_CONTINUE = 1; - - /** - * Error severity: message, plus full stop, critical error reached. - * - * @var int - */ - const STOP_CRITICAL = 2; - - /** - * The SMTP standard CRLF line break. - * If you want to change line break format, change static::$LE, not this. - */ - const CRLF = "\r\n"; - - /** - * "Folding White Space" a white space string used for line folding. - */ - const FWS = ' '; - - /** - * SMTP RFC standard line ending; Carriage Return, Line Feed. - * - * @var string - */ - protected static $LE = self::CRLF; - - /** - * The maximum line length supported by mail(). - * - * Background: mail() will sometimes corrupt messages - * with headers longer than 65 chars, see #818. - * - * @var int - */ - const MAIL_MAX_LINE_LENGTH = 63; - - /** - * The maximum line length allowed by RFC 2822 section 2.1.1. - * - * @var int - */ - const MAX_LINE_LENGTH = 998; - - /** - * The lower maximum line length allowed by RFC 2822 section 2.1.1. - * This length does NOT include the line break - * 76 means that lines will be 77 or 78 chars depending on whether - * the line break format is LF or CRLF; both are valid. - * - * @var int - */ - const STD_LINE_LENGTH = 76; - - /** - * Constructor. - * - * @param bool $exceptions Should we throw external exceptions? - */ - public function __construct($exceptions = null) - { - if (null !== $exceptions) { - $this->exceptions = (bool) $exceptions; - } - //Pick an appropriate debug output format automatically - $this->Debugoutput = (strpos(PHP_SAPI, 'cli') !== false ? 'echo' : 'html'); - } - - /** - * Destructor. - */ - public function __destruct() - { - //Close any open SMTP connection nicely - $this->smtpClose(); - } - - /** - * Call mail() in a safe_mode-aware fashion. - * Also, unless sendmail_path points to sendmail (or something that - * claims to be sendmail), don't pass params (not a perfect fix, - * but it will do). - * - * @param string $to To - * @param string $subject Subject - * @param string $body Message Body - * @param string $header Additional Header(s) - * @param string|null $params Params - * - * @return bool - */ - private function mailPassthru($to, $subject, $body, $header, $params) - { - //Check overloading of mail function to avoid double-encoding - if ((int)ini_get('mbstring.func_overload') & 1) { // phpcs:ignore PHPCompatibility.IniDirectives.RemovedIniDirectives.mbstring_func_overloadDeprecated - $subject = $this->secureHeader($subject); - } else { - $subject = $this->encodeHeader($this->secureHeader($subject)); - } - //Calling mail() with null params breaks - $this->edebug('Sending with mail()'); - $this->edebug('Sendmail path: ' . ini_get('sendmail_path')); - $this->edebug("Envelope sender: {$this->Sender}"); - $this->edebug("To: {$to}"); - $this->edebug("Subject: {$subject}"); - $this->edebug("Headers: {$header}"); - if (!$this->UseSendmailOptions || null === $params) { - $result = @mail($to, $subject, $body, $header); - } else { - $this->edebug("Additional params: {$params}"); - $result = @mail($to, $subject, $body, $header, $params); - } - $this->edebug('Result: ' . ($result ? 'true' : 'false')); - return $result; - } - - /** - * Output debugging info via a user-defined method. - * Only generates output if debug output is enabled. - * - * @see PHPMailer::$Debugoutput - * @see PHPMailer::$SMTPDebug - * - * @param string $str - */ - protected function edebug($str) - { - if ($this->SMTPDebug <= 0) { - return; - } - //Is this a PSR-3 logger? - if ($this->Debugoutput instanceof \Psr\Log\LoggerInterface) { - $this->Debugoutput->debug(rtrim($str, "\r\n")); - - return; - } - //Avoid clash with built-in function names - if (is_callable($this->Debugoutput) && !in_array($this->Debugoutput, ['error_log', 'html', 'echo'])) { - call_user_func($this->Debugoutput, $str, $this->SMTPDebug); - - return; - } - switch ($this->Debugoutput) { - case 'error_log': - //Don't output, just log - /** @noinspection ForgottenDebugOutputInspection */ - error_log($str); - break; - case 'html': - //Cleans up output a bit for a better looking, HTML-safe output - echo htmlentities( - preg_replace('/[\r\n]+/', '', $str), - ENT_QUOTES, - 'UTF-8' - ), "
\n"; - break; - case 'echo': - default: - //Normalize line breaks - $str = preg_replace('/\r\n|\r/m', "\n", $str); - echo gmdate('Y-m-d H:i:s'), - "\t", - //Trim trailing space - trim( - //Indent for readability, except for trailing break - str_replace( - "\n", - "\n \t ", - trim($str) - ) - ), - "\n"; - } - } - - /** - * Sets message type to HTML or plain. - * - * @param bool $isHtml True for HTML mode - */ - public function isHTML($isHtml = true) - { - if ($isHtml) { - $this->ContentType = static::CONTENT_TYPE_TEXT_HTML; - } else { - $this->ContentType = static::CONTENT_TYPE_PLAINTEXT; - } - } - - /** - * Send messages using SMTP. - */ - public function isSMTP() - { - $this->Mailer = 'smtp'; - } - - /** - * Send messages using PHP's mail() function. - */ - public function isMail() - { - $this->Mailer = 'mail'; - } - - /** - * Send messages using $Sendmail. - */ - public function isSendmail() - { - $ini_sendmail_path = ini_get('sendmail_path'); - - if (false === stripos($ini_sendmail_path, 'sendmail')) { - $this->Sendmail = '/usr/sbin/sendmail'; - } else { - $this->Sendmail = $ini_sendmail_path; - } - $this->Mailer = 'sendmail'; - } - - /** - * Send messages using qmail. - */ - public function isQmail() - { - $ini_sendmail_path = ini_get('sendmail_path'); - - if (false === stripos($ini_sendmail_path, 'qmail')) { - $this->Sendmail = '/var/qmail/bin/qmail-inject'; - } else { - $this->Sendmail = $ini_sendmail_path; - } - $this->Mailer = 'qmail'; - } - - /** - * Add a "To" address. - * - * @param string $address The email address to send to - * @param string $name - * - * @throws Exception - * - * @return bool true on success, false if address already used or invalid in some way - */ - public function addAddress($address, $name = '') - { - return $this->addOrEnqueueAnAddress('to', $address, $name); - } - - /** - * Add a "CC" address. - * - * @param string $address The email address to send to - * @param string $name - * - * @throws Exception - * - * @return bool true on success, false if address already used or invalid in some way - */ - public function addCC($address, $name = '') - { - return $this->addOrEnqueueAnAddress('cc', $address, $name); - } - - /** - * Add a "BCC" address. - * - * @param string $address The email address to send to - * @param string $name - * - * @throws Exception - * - * @return bool true on success, false if address already used or invalid in some way - */ - public function addBCC($address, $name = '') - { - return $this->addOrEnqueueAnAddress('bcc', $address, $name); - } - - /** - * Add a "Reply-To" address. - * - * @param string $address The email address to reply to - * @param string $name - * - * @throws Exception - * - * @return bool true on success, false if address already used or invalid in some way - */ - public function addReplyTo($address, $name = '') - { - return $this->addOrEnqueueAnAddress('Reply-To', $address, $name); - } - - /** - * Add an address to one of the recipient arrays or to the ReplyTo array. Because PHPMailer - * can't validate addresses with an IDN without knowing the PHPMailer::$CharSet (that can still - * be modified after calling this function), addition of such addresses is delayed until send(). - * Addresses that have been added already return false, but do not throw exceptions. - * - * @param string $kind One of 'to', 'cc', 'bcc', or 'Reply-To' - * @param string $address The email address - * @param string $name An optional username associated with the address - * - * @throws Exception - * - * @return bool true on success, false if address already used or invalid in some way - */ - protected function addOrEnqueueAnAddress($kind, $address, $name) - { - $pos = false; - if ($address !== null) { - $address = trim($address); - $pos = strrpos($address, '@'); - } - if (false === $pos) { - //At-sign is missing. - $error_message = sprintf( - '%s (%s): %s', - $this->lang('invalid_address'), - $kind, - $address - ); - $this->setError($error_message); - $this->edebug($error_message); - if ($this->exceptions) { - throw new Exception($error_message); - } - - return false; - } - if ($name !== null && is_string($name)) { - $name = trim(preg_replace('/[\r\n]+/', '', $name)); //Strip breaks and trim - } else { - $name = ''; - } - $params = [$kind, $address, $name]; - //Enqueue addresses with IDN until we know the PHPMailer::$CharSet. - //Domain is assumed to be whatever is after the last @ symbol in the address - if (static::idnSupported() && $this->has8bitChars(substr($address, ++$pos))) { - if ('Reply-To' !== $kind) { - if (!array_key_exists($address, $this->RecipientsQueue)) { - $this->RecipientsQueue[$address] = $params; - - return true; - } - } elseif (!array_key_exists($address, $this->ReplyToQueue)) { - $this->ReplyToQueue[$address] = $params; - - return true; - } - - return false; - } - - //Immediately add standard addresses without IDN. - return call_user_func_array([$this, 'addAnAddress'], $params); - } - - /** - * Set the boundaries to use for delimiting MIME parts. - * If you override this, ensure you set all 3 boundaries to unique values. - * The default boundaries include a "=_" sequence which cannot occur in quoted-printable bodies, - * as suggested by https://www.rfc-editor.org/rfc/rfc2045#section-6.7 - * - * @return void - */ - public function setBoundaries() - { - $this->uniqueid = $this->generateId(); - $this->boundary[1] = 'b1=_' . $this->uniqueid; - $this->boundary[2] = 'b2=_' . $this->uniqueid; - $this->boundary[3] = 'b3=_' . $this->uniqueid; - } - - /** - * Add an address to one of the recipient arrays or to the ReplyTo array. - * Addresses that have been added already return false, but do not throw exceptions. - * - * @param string $kind One of 'to', 'cc', 'bcc', or 'ReplyTo' - * @param string $address The email address to send, resp. to reply to - * @param string $name - * - * @throws Exception - * - * @return bool true on success, false if address already used or invalid in some way - */ - protected function addAnAddress($kind, $address, $name = '') - { - if (!in_array($kind, ['to', 'cc', 'bcc', 'Reply-To'])) { - $error_message = sprintf( - '%s: %s', - $this->lang('Invalid recipient kind'), - $kind - ); - $this->setError($error_message); - $this->edebug($error_message); - if ($this->exceptions) { - throw new Exception($error_message); - } - - return false; - } - if (!static::validateAddress($address)) { - $error_message = sprintf( - '%s (%s): %s', - $this->lang('invalid_address'), - $kind, - $address - ); - $this->setError($error_message); - $this->edebug($error_message); - if ($this->exceptions) { - throw new Exception($error_message); - } - - return false; - } - if ('Reply-To' !== $kind) { - if (!array_key_exists(strtolower($address), $this->all_recipients)) { - $this->{$kind}[] = [$address, $name]; - $this->all_recipients[strtolower($address)] = true; - - return true; - } - } elseif (!array_key_exists(strtolower($address), $this->ReplyTo)) { - $this->ReplyTo[strtolower($address)] = [$address, $name]; - - return true; - } - - return false; - } - - /** - * Parse and validate a string containing one or more RFC822-style comma-separated email addresses - * of the form "display name
" into an array of name/address pairs. - * Uses the imap_rfc822_parse_adrlist function if the IMAP extension is available. - * Note that quotes in the name part are removed. - * - * @see https://www.andrew.cmu.edu/user/agreen1/testing/mrbs/web/Mail/RFC822.php A more careful implementation - * - * @param string $addrstr The address list string - * @param bool $useimap Whether to use the IMAP extension to parse the list - * @param string $charset The charset to use when decoding the address list string. - * - * @return array - */ - public static function parseAddresses($addrstr, $useimap = true, $charset = self::CHARSET_ISO88591) - { - $addresses = []; - if ($useimap && function_exists('imap_rfc822_parse_adrlist')) { - //Use this built-in parser if it's available - $list = imap_rfc822_parse_adrlist($addrstr, ''); - // Clear any potential IMAP errors to get rid of notices being thrown at end of script. - imap_errors(); - foreach ($list as $address) { - if ( - '.SYNTAX-ERROR.' !== $address->host && - static::validateAddress($address->mailbox . '@' . $address->host) - ) { - //Decode the name part if it's present and encoded - if ( - property_exists($address, 'personal') && - //Check for a Mbstring constant rather than using extension_loaded, which is sometimes disabled - defined('MB_CASE_UPPER') && - preg_match('/^=\?.*\?=$/s', $address->personal) - ) { - $origCharset = mb_internal_encoding(); - mb_internal_encoding($charset); - //Undo any RFC2047-encoded spaces-as-underscores - $address->personal = str_replace('_', '=20', $address->personal); - //Decode the name - $address->personal = mb_decode_mimeheader($address->personal); - mb_internal_encoding($origCharset); - } - - $addresses[] = [ - 'name' => (property_exists($address, 'personal') ? $address->personal : ''), - 'address' => $address->mailbox . '@' . $address->host, - ]; - } - } - } else { - //Use this simpler parser - $list = explode(',', $addrstr); - foreach ($list as $address) { - $address = trim($address); - //Is there a separate name part? - if (strpos($address, '<') === false) { - //No separate name, just use the whole thing - if (static::validateAddress($address)) { - $addresses[] = [ - 'name' => '', - 'address' => $address, - ]; - } - } else { - list($name, $email) = explode('<', $address); - $email = trim(str_replace('>', '', $email)); - $name = trim($name); - if (static::validateAddress($email)) { - //Check for a Mbstring constant rather than using extension_loaded, which is sometimes disabled - //If this name is encoded, decode it - if (defined('MB_CASE_UPPER') && preg_match('/^=\?.*\?=$/s', $name)) { - $origCharset = mb_internal_encoding(); - mb_internal_encoding($charset); - //Undo any RFC2047-encoded spaces-as-underscores - $name = str_replace('_', '=20', $name); - //Decode the name - $name = mb_decode_mimeheader($name); - mb_internal_encoding($origCharset); - } - $addresses[] = [ - //Remove any surrounding quotes and spaces from the name - 'name' => trim($name, '\'" '), - 'address' => $email, - ]; - } - } - } - } - - return $addresses; - } - - /** - * Set the From and FromName properties. - * - * @param string $address - * @param string $name - * @param bool $auto Whether to also set the Sender address, defaults to true - * - * @throws Exception - * - * @return bool - */ - public function setFrom($address, $name = '', $auto = true) - { - $address = trim((string)$address); - $name = trim(preg_replace('/[\r\n]+/', '', $name)); //Strip breaks and trim - //Don't validate now addresses with IDN. Will be done in send(). - $pos = strrpos($address, '@'); - if ( - (false === $pos) - || ((!$this->has8bitChars(substr($address, ++$pos)) || !static::idnSupported()) - && !static::validateAddress($address)) - ) { - $error_message = sprintf( - '%s (From): %s', - $this->lang('invalid_address'), - $address - ); - $this->setError($error_message); - $this->edebug($error_message); - if ($this->exceptions) { - throw new Exception($error_message); - } - - return false; - } - $this->From = $address; - $this->FromName = $name; - if ($auto && empty($this->Sender)) { - $this->Sender = $address; - } - - return true; - } - - /** - * Return the Message-ID header of the last email. - * Technically this is the value from the last time the headers were created, - * but it's also the message ID of the last sent message except in - * pathological cases. - * - * @return string - */ - public function getLastMessageID() - { - return $this->lastMessageID; - } - - /** - * Check that a string looks like an email address. - * Validation patterns supported: - * * `auto` Pick best pattern automatically; - * * `pcre8` Use the squiloople.com pattern, requires PCRE > 8.0; - * * `pcre` Use old PCRE implementation; - * * `php` Use PHP built-in FILTER_VALIDATE_EMAIL; - * * `html5` Use the pattern given by the HTML5 spec for 'email' type form input elements. - * * `noregex` Don't use a regex: super fast, really dumb. - * Alternatively you may pass in a callable to inject your own validator, for example: - * - * ```php - * PHPMailer::validateAddress('user@example.com', function($address) { - * return (strpos($address, '@') !== false); - * }); - * ``` - * - * You can also set the PHPMailer::$validator static to a callable, allowing built-in methods to use your validator. - * - * @param string $address The email address to check - * @param string|callable $patternselect Which pattern to use - * - * @return bool - */ - public static function validateAddress($address, $patternselect = null) - { - if (null === $patternselect) { - $patternselect = static::$validator; - } - //Don't allow strings as callables, see SECURITY.md and CVE-2021-3603 - if (is_callable($patternselect) && !is_string($patternselect)) { - return call_user_func($patternselect, $address); - } - //Reject line breaks in addresses; it's valid RFC5322, but not RFC5321 - if (strpos($address, "\n") !== false || strpos($address, "\r") !== false) { - return false; - } - switch ($patternselect) { - case 'pcre': //Kept for BC - case 'pcre8': - /* - * A more complex and more permissive version of the RFC5322 regex on which FILTER_VALIDATE_EMAIL - * is based. - * In addition to the addresses allowed by filter_var, also permits: - * * dotless domains: `a@b` - * * comments: `1234 @ local(blah) .machine .example` - * * quoted elements: `'"test blah"@example.org'` - * * numeric TLDs: `a@b.123` - * * unbracketed IPv4 literals: `a@192.168.0.1` - * * IPv6 literals: 'first.last@[IPv6:a1::]' - * Not all of these will necessarily work for sending! - * - * @copyright 2009-2010 Michael Rushton - * Feel free to use and redistribute this code. But please keep this copyright notice. - */ - return (bool) preg_match( - '/^(?!(?>(?1)"?(?>\\\[ -~]|[^"])"?(?1)){255,})(?!(?>(?1)"?(?>\\\[ -~]|[^"])"?(?1)){65,}@)' . - '((?>(?>(?>((?>(?>(?>\x0D\x0A)?[\t ])+|(?>[\t ]*\x0D\x0A)?[\t ]+)?)(\((?>(?2)' . - '(?>[\x01-\x08\x0B\x0C\x0E-\'*-\[\]-\x7F]|\\\[\x00-\x7F]|(?3)))*(?2)\)))+(?2))|(?2))?)' . - '([!#-\'*+\/-9=?^-~-]+|"(?>(?2)(?>[\x01-\x08\x0B\x0C\x0E-!#-\[\]-\x7F]|\\\[\x00-\x7F]))*' . - '(?2)")(?>(?1)\.(?1)(?4))*(?1)@(?!(?1)[a-z0-9-]{64,})(?1)(?>([a-z0-9](?>[a-z0-9-]*[a-z0-9])?)' . - '(?>(?1)\.(?!(?1)[a-z0-9-]{64,})(?1)(?5)){0,126}|\[(?:(?>IPv6:(?>([a-f0-9]{1,4})(?>:(?6)){7}' . - '|(?!(?:.*[a-f0-9][:\]]){8,})((?6)(?>:(?6)){0,6})?::(?7)?))|(?>(?>IPv6:(?>(?6)(?>:(?6)){5}:' . - '|(?!(?:.*[a-f0-9]:){6,})(?8)?::(?>((?6)(?>:(?6)){0,4}):)?))?(25[0-5]|2[0-4][0-9]|1[0-9]{2}' . - '|[1-9]?[0-9])(?>\.(?9)){3}))\])(?1)$/isD', - $address - ); - case 'html5': - /* - * This is the pattern used in the HTML5 spec for validation of 'email' type form input elements. - * - * @see https://html.spec.whatwg.org/#e-mail-state-(type=email) - */ - return (bool) preg_match( - '/^[a-zA-Z0-9.!#$%&\'*+\/=?^_`{|}~-]+@[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}' . - '[a-zA-Z0-9])?(?:\.[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?)*$/sD', - $address - ); - case 'php': - default: - return filter_var($address, FILTER_VALIDATE_EMAIL) !== false; - } - } - - /** - * Tells whether IDNs (Internationalized Domain Names) are supported or not. This requires the - * `intl` and `mbstring` PHP extensions. - * - * @return bool `true` if required functions for IDN support are present - */ - public static function idnSupported() - { - return function_exists('idn_to_ascii') && function_exists('mb_convert_encoding'); - } - - /** - * Converts IDN in given email address to its ASCII form, also known as punycode, if possible. - * Important: Address must be passed in same encoding as currently set in PHPMailer::$CharSet. - * This function silently returns unmodified address if: - * - No conversion is necessary (i.e. domain name is not an IDN, or is already in ASCII form) - * - Conversion to punycode is impossible (e.g. required PHP functions are not available) - * or fails for any reason (e.g. domain contains characters not allowed in an IDN). - * - * @see PHPMailer::$CharSet - * - * @param string $address The email address to convert - * - * @return string The encoded address in ASCII form - */ - public function punyencodeAddress($address) - { - //Verify we have required functions, CharSet, and at-sign. - $pos = strrpos($address, '@'); - if ( - !empty($this->CharSet) && - false !== $pos && - static::idnSupported() - ) { - $domain = substr($address, ++$pos); - //Verify CharSet string is a valid one, and domain properly encoded in this CharSet. - if ($this->has8bitChars($domain) && @mb_check_encoding($domain, $this->CharSet)) { - //Convert the domain from whatever charset it's in to UTF-8 - $domain = mb_convert_encoding($domain, self::CHARSET_UTF8, $this->CharSet); - //Ignore IDE complaints about this line - method signature changed in PHP 5.4 - $errorcode = 0; - if (defined('INTL_IDNA_VARIANT_UTS46')) { - //Use the current punycode standard (appeared in PHP 7.2) - $punycode = idn_to_ascii( - $domain, - \IDNA_DEFAULT | \IDNA_USE_STD3_RULES | \IDNA_CHECK_BIDI | - \IDNA_CHECK_CONTEXTJ | \IDNA_NONTRANSITIONAL_TO_ASCII, - \INTL_IDNA_VARIANT_UTS46 - ); - } elseif (defined('INTL_IDNA_VARIANT_2003')) { - //Fall back to this old, deprecated/removed encoding - // phpcs:ignore PHPCompatibility.Constants.RemovedConstants.intl_idna_variant_2003Deprecated - $punycode = idn_to_ascii($domain, $errorcode, \INTL_IDNA_VARIANT_2003); - } else { - //Fall back to a default we don't know about - // phpcs:ignore PHPCompatibility.ParameterValues.NewIDNVariantDefault.NotSet - $punycode = idn_to_ascii($domain, $errorcode); - } - if (false !== $punycode) { - return substr($address, 0, $pos) . $punycode; - } - } - } - - return $address; - } - - /** - * Create a message and send it. - * Uses the sending method specified by $Mailer. - * - * @throws Exception - * - * @return bool false on error - See the ErrorInfo property for details of the error - */ - public function send() - { - try { - if (!$this->preSend()) { - return false; - } - - return $this->postSend(); - } catch (Exception $exc) { - $this->mailHeader = ''; - $this->setError($exc->getMessage()); - if ($this->exceptions) { - throw $exc; - } - - return false; - } - } - - /** - * Prepare a message for sending. - * - * @throws Exception - * - * @return bool - */ - public function preSend() - { - if ( - 'smtp' === $this->Mailer - || ('mail' === $this->Mailer && (\PHP_VERSION_ID >= 80000 || stripos(PHP_OS, 'WIN') === 0)) - ) { - //SMTP mandates RFC-compliant line endings - //and it's also used with mail() on Windows - static::setLE(self::CRLF); - } else { - //Maintain backward compatibility with legacy Linux command line mailers - static::setLE(PHP_EOL); - } - //Check for buggy PHP versions that add a header with an incorrect line break - if ( - 'mail' === $this->Mailer - && ((\PHP_VERSION_ID >= 70000 && \PHP_VERSION_ID < 70017) - || (\PHP_VERSION_ID >= 70100 && \PHP_VERSION_ID < 70103)) - && ini_get('mail.add_x_header') === '1' - && stripos(PHP_OS, 'WIN') === 0 - ) { - trigger_error($this->lang('buggy_php'), E_USER_WARNING); - } - - try { - $this->error_count = 0; //Reset errors - $this->mailHeader = ''; - - //Dequeue recipient and Reply-To addresses with IDN - foreach (array_merge($this->RecipientsQueue, $this->ReplyToQueue) as $params) { - $params[1] = $this->punyencodeAddress($params[1]); - call_user_func_array([$this, 'addAnAddress'], $params); - } - if (count($this->to) + count($this->cc) + count($this->bcc) < 1) { - throw new Exception($this->lang('provide_address'), self::STOP_CRITICAL); - } - - //Validate From, Sender, and ConfirmReadingTo addresses - foreach (['From', 'Sender', 'ConfirmReadingTo'] as $address_kind) { - if ($this->{$address_kind} === null) { - $this->{$address_kind} = ''; - continue; - } - $this->{$address_kind} = trim($this->{$address_kind}); - if (empty($this->{$address_kind})) { - continue; - } - $this->{$address_kind} = $this->punyencodeAddress($this->{$address_kind}); - if (!static::validateAddress($this->{$address_kind})) { - $error_message = sprintf( - '%s (%s): %s', - $this->lang('invalid_address'), - $address_kind, - $this->{$address_kind} - ); - $this->setError($error_message); - $this->edebug($error_message); - if ($this->exceptions) { - throw new Exception($error_message); - } - - return false; - } - } - - //Set whether the message is multipart/alternative - if ($this->alternativeExists()) { - $this->ContentType = static::CONTENT_TYPE_MULTIPART_ALTERNATIVE; - } - - $this->setMessageType(); - //Refuse to send an empty message unless we are specifically allowing it - if (!$this->AllowEmpty && empty($this->Body)) { - throw new Exception($this->lang('empty_message'), self::STOP_CRITICAL); - } - - //Trim subject consistently - $this->Subject = trim($this->Subject); - //Create body before headers in case body makes changes to headers (e.g. altering transfer encoding) - $this->MIMEHeader = ''; - $this->MIMEBody = $this->createBody(); - //createBody may have added some headers, so retain them - $tempheaders = $this->MIMEHeader; - $this->MIMEHeader = $this->createHeader(); - $this->MIMEHeader .= $tempheaders; - - //To capture the complete message when using mail(), create - //an extra header list which createHeader() doesn't fold in - if ('mail' === $this->Mailer) { - if (count($this->to) > 0) { - $this->mailHeader .= $this->addrAppend('To', $this->to); - } else { - $this->mailHeader .= $this->headerLine('To', 'undisclosed-recipients:;'); - } - $this->mailHeader .= $this->headerLine( - 'Subject', - $this->encodeHeader($this->secureHeader($this->Subject)) - ); - } - - //Sign with DKIM if enabled - if ( - !empty($this->DKIM_domain) - && !empty($this->DKIM_selector) - && (!empty($this->DKIM_private_string) - || (!empty($this->DKIM_private) - && static::isPermittedPath($this->DKIM_private) - && file_exists($this->DKIM_private) - ) - ) - ) { - $header_dkim = $this->DKIM_Add( - $this->MIMEHeader . $this->mailHeader, - $this->encodeHeader($this->secureHeader($this->Subject)), - $this->MIMEBody - ); - $this->MIMEHeader = static::stripTrailingWSP($this->MIMEHeader) . static::$LE . - static::normalizeBreaks($header_dkim) . static::$LE; - } - - return true; - } catch (Exception $exc) { - $this->setError($exc->getMessage()); - if ($this->exceptions) { - throw $exc; - } - - return false; - } - } - - /** - * Actually send a message via the selected mechanism. - * - * @throws Exception - * - * @return bool - */ - public function postSend() - { - try { - //Choose the mailer and send through it - switch ($this->Mailer) { - case 'sendmail': - case 'qmail': - return $this->sendmailSend($this->MIMEHeader, $this->MIMEBody); - case 'smtp': - return $this->smtpSend($this->MIMEHeader, $this->MIMEBody); - case 'mail': - return $this->mailSend($this->MIMEHeader, $this->MIMEBody); - default: - $sendMethod = $this->Mailer . 'Send'; - if (method_exists($this, $sendMethod)) { - return $this->{$sendMethod}($this->MIMEHeader, $this->MIMEBody); - } - - return $this->mailSend($this->MIMEHeader, $this->MIMEBody); - } - } catch (Exception $exc) { - $this->setError($exc->getMessage()); - $this->edebug($exc->getMessage()); - if ($this->Mailer === 'smtp' && $this->SMTPKeepAlive == true && $this->smtp->connected()) { - $this->smtp->reset(); - } - if ($this->exceptions) { - throw $exc; - } - } - - return false; - } - - /** - * Send mail using the $Sendmail program. - * - * @see PHPMailer::$Sendmail - * - * @param string $header The message headers - * @param string $body The message body - * - * @throws Exception - * - * @return bool - */ - protected function sendmailSend($header, $body) - { - if ($this->Mailer === 'qmail') { - $this->edebug('Sending with qmail'); - } else { - $this->edebug('Sending with sendmail'); - } - $header = static::stripTrailingWSP($header) . static::$LE . static::$LE; - //This sets the SMTP envelope sender which gets turned into a return-path header by the receiver - //A space after `-f` is optional, but there is a long history of its presence - //causing problems, so we don't use one - //Exim docs: https://www.exim.org/exim-html-current/doc/html/spec_html/ch-the_exim_command_line.html - //Sendmail docs: https://www.sendmail.org/~ca/email/man/sendmail.html - //Example problem: https://www.drupal.org/node/1057954 - - //PHP 5.6 workaround - $sendmail_from_value = ini_get('sendmail_from'); - if (empty($this->Sender) && !empty($sendmail_from_value)) { - //PHP config has a sender address we can use - $this->Sender = ini_get('sendmail_from'); - } - //CVE-2016-10033, CVE-2016-10045: Don't pass -f if characters will be escaped. - if (!empty($this->Sender) && static::validateAddress($this->Sender) && self::isShellSafe($this->Sender)) { - if ($this->Mailer === 'qmail') { - $sendmailFmt = '%s -f%s'; - } else { - $sendmailFmt = '%s -oi -f%s -t'; - } - } else { - //allow sendmail to choose a default envelope sender. It may - //seem preferable to force it to use the From header as with - //SMTP, but that introduces new problems (see - //), and - //it has historically worked this way. - $sendmailFmt = '%s -oi -t'; - } - - $sendmail = sprintf($sendmailFmt, escapeshellcmd($this->Sendmail), $this->Sender); - $this->edebug('Sendmail path: ' . $this->Sendmail); - $this->edebug('Sendmail command: ' . $sendmail); - $this->edebug('Envelope sender: ' . $this->Sender); - $this->edebug("Headers: {$header}"); - - if ($this->SingleTo) { - foreach ($this->SingleToArray as $toAddr) { - $mail = @popen($sendmail, 'w'); - if (!$mail) { - throw new Exception($this->lang('execute') . $this->Sendmail, self::STOP_CRITICAL); - } - $this->edebug("To: {$toAddr}"); - fwrite($mail, 'To: ' . $toAddr . "\n"); - fwrite($mail, $header); - fwrite($mail, $body); - $result = pclose($mail); - $addrinfo = static::parseAddresses($toAddr, true, $this->CharSet); - $this->doCallback( - ($result === 0), - [[$addrinfo['address'], $addrinfo['name']]], - $this->cc, - $this->bcc, - $this->Subject, - $body, - $this->From, - [] - ); - $this->edebug("Result: " . ($result === 0 ? 'true' : 'false')); - if (0 !== $result) { - throw new Exception($this->lang('execute') . $this->Sendmail, self::STOP_CRITICAL); - } - } - } else { - $mail = @popen($sendmail, 'w'); - if (!$mail) { - throw new Exception($this->lang('execute') . $this->Sendmail, self::STOP_CRITICAL); - } - fwrite($mail, $header); - fwrite($mail, $body); - $result = pclose($mail); - $this->doCallback( - ($result === 0), - $this->to, - $this->cc, - $this->bcc, - $this->Subject, - $body, - $this->From, - [] - ); - $this->edebug("Result: " . ($result === 0 ? 'true' : 'false')); - if (0 !== $result) { - throw new Exception($this->lang('execute') . $this->Sendmail, self::STOP_CRITICAL); - } - } - - return true; - } - - /** - * Fix CVE-2016-10033 and CVE-2016-10045 by disallowing potentially unsafe shell characters. - * Note that escapeshellarg and escapeshellcmd are inadequate for our purposes, especially on Windows. - * - * @see https://github.com/PHPMailer/PHPMailer/issues/924 CVE-2016-10045 bug report - * - * @param string $string The string to be validated - * - * @return bool - */ - protected static function isShellSafe($string) - { - //It's not possible to use shell commands safely (which includes the mail() function) without escapeshellarg, - //but some hosting providers disable it, creating a security problem that we don't want to have to deal with, - //so we don't. - if (!function_exists('escapeshellarg') || !function_exists('escapeshellcmd')) { - return false; - } - - if ( - escapeshellcmd($string) !== $string - || !in_array(escapeshellarg($string), ["'$string'", "\"$string\""]) - ) { - return false; - } - - $length = strlen($string); - - for ($i = 0; $i < $length; ++$i) { - $c = $string[$i]; - - //All other characters have a special meaning in at least one common shell, including = and +. - //Full stop (.) has a special meaning in cmd.exe, but its impact should be negligible here. - //Note that this does permit non-Latin alphanumeric characters based on the current locale. - if (!ctype_alnum($c) && strpos('@_-.', $c) === false) { - return false; - } - } - - return true; - } - - /** - * Check whether a file path is of a permitted type. - * Used to reject URLs and phar files from functions that access local file paths, - * such as addAttachment. - * - * @param string $path A relative or absolute path to a file - * - * @return bool - */ - protected static function isPermittedPath($path) - { - //Matches scheme definition from https://www.rfc-editor.org/rfc/rfc3986#section-3.1 - return !preg_match('#^[a-z][a-z\d+.-]*://#i', $path); - } - - /** - * Check whether a file path is safe, accessible, and readable. - * - * @param string $path A relative or absolute path to a file - * - * @return bool - */ - protected static function fileIsAccessible($path) - { - if (!static::isPermittedPath($path)) { - return false; - } - $readable = is_file($path); - //If not a UNC path (expected to start with \\), check read permission, see #2069 - if (strpos($path, '\\\\') !== 0) { - $readable = $readable && is_readable($path); - } - return $readable; - } - - /** - * Send mail using the PHP mail() function. - * - * @see https://www.php.net/manual/en/book.mail.php - * - * @param string $header The message headers - * @param string $body The message body - * - * @throws Exception - * - * @return bool - */ - protected function mailSend($header, $body) - { - $header = static::stripTrailingWSP($header) . static::$LE . static::$LE; - - $toArr = []; - foreach ($this->to as $toaddr) { - $toArr[] = $this->addrFormat($toaddr); - } - $to = trim(implode(', ', $toArr)); - - //If there are no To-addresses (e.g. when sending only to BCC-addresses) - //the following should be added to get a correct DKIM-signature. - //Compare with $this->preSend() - if ($to === '') { - $to = 'undisclosed-recipients:;'; - } - - $params = null; - //This sets the SMTP envelope sender which gets turned into a return-path header by the receiver - //A space after `-f` is optional, but there is a long history of its presence - //causing problems, so we don't use one - //Exim docs: https://www.exim.org/exim-html-current/doc/html/spec_html/ch-the_exim_command_line.html - //Sendmail docs: https://www.sendmail.org/~ca/email/man/sendmail.html - //Example problem: https://www.drupal.org/node/1057954 - //CVE-2016-10033, CVE-2016-10045: Don't pass -f if characters will be escaped. - - //PHP 5.6 workaround - $sendmail_from_value = ini_get('sendmail_from'); - if (empty($this->Sender) && !empty($sendmail_from_value)) { - //PHP config has a sender address we can use - $this->Sender = ini_get('sendmail_from'); - } - if (!empty($this->Sender) && static::validateAddress($this->Sender)) { - if (self::isShellSafe($this->Sender)) { - $params = sprintf('-f%s', $this->Sender); - } - $old_from = ini_get('sendmail_from'); - ini_set('sendmail_from', $this->Sender); - } - $result = false; - if ($this->SingleTo && count($toArr) > 1) { - foreach ($toArr as $toAddr) { - $result = $this->mailPassthru($toAddr, $this->Subject, $body, $header, $params); - $addrinfo = static::parseAddresses($toAddr, true, $this->CharSet); - $this->doCallback( - $result, - [[$addrinfo['address'], $addrinfo['name']]], - $this->cc, - $this->bcc, - $this->Subject, - $body, - $this->From, - [] - ); - } - } else { - $result = $this->mailPassthru($to, $this->Subject, $body, $header, $params); - $this->doCallback($result, $this->to, $this->cc, $this->bcc, $this->Subject, $body, $this->From, []); - } - if (isset($old_from)) { - ini_set('sendmail_from', $old_from); - } - if (!$result) { - throw new Exception($this->lang('instantiate'), self::STOP_CRITICAL); - } - - return true; - } - - /** - * Get an instance to use for SMTP operations. - * Override this function to load your own SMTP implementation, - * or set one with setSMTPInstance. - * - * @return SMTP - */ - public function getSMTPInstance() - { - if (!is_object($this->smtp)) { - $this->smtp = new SMTP(); - } - - return $this->smtp; - } - - /** - * Provide an instance to use for SMTP operations. - * - * @return SMTP - */ - public function setSMTPInstance(SMTP $smtp) - { - $this->smtp = $smtp; - - return $this->smtp; - } - - /** - * Provide SMTP XCLIENT attributes - * - * @param string $name Attribute name - * @param ?string $value Attribute value - * - * @return bool - */ - public function setSMTPXclientAttribute($name, $value) - { - if (!in_array($name, SMTP::$xclient_allowed_attributes)) { - return false; - } - if (isset($this->SMTPXClient[$name]) && $value === null) { - unset($this->SMTPXClient[$name]); - } elseif ($value !== null) { - $this->SMTPXClient[$name] = $value; - } - - return true; - } - - /** - * Get SMTP XCLIENT attributes - * - * @return array - */ - public function getSMTPXclientAttributes() - { - return $this->SMTPXClient; - } - - /** - * Send mail via SMTP. - * Returns false if there is a bad MAIL FROM, RCPT, or DATA input. - * - * @see PHPMailer::setSMTPInstance() to use a different class. - * - * @uses \PHPMailer\PHPMailer\SMTP - * - * @param string $header The message headers - * @param string $body The message body - * - * @throws Exception - * - * @return bool - */ - protected function smtpSend($header, $body) - { - $header = static::stripTrailingWSP($header) . static::$LE . static::$LE; - $bad_rcpt = []; - if (!$this->smtpConnect($this->SMTPOptions)) { - throw new Exception($this->lang('smtp_connect_failed'), self::STOP_CRITICAL); - } - //Sender already validated in preSend() - if ('' === $this->Sender) { - $smtp_from = $this->From; - } else { - $smtp_from = $this->Sender; - } - if (count($this->SMTPXClient)) { - $this->smtp->xclient($this->SMTPXClient); - } - if (!$this->smtp->mail($smtp_from)) { - $this->setError($this->lang('from_failed') . $smtp_from . ' : ' . implode(',', $this->smtp->getError())); - throw new Exception($this->ErrorInfo, self::STOP_CRITICAL); - } - - $callbacks = []; - //Attempt to send to all recipients - foreach ([$this->to, $this->cc, $this->bcc] as $togroup) { - foreach ($togroup as $to) { - if (!$this->smtp->recipient($to[0], $this->dsn)) { - $error = $this->smtp->getError(); - $bad_rcpt[] = ['to' => $to[0], 'error' => $error['detail']]; - $isSent = false; - } else { - $isSent = true; - } - - $callbacks[] = ['issent' => $isSent, 'to' => $to[0], 'name' => $to[1]]; - } - } - - //Only send the DATA command if we have viable recipients - if ((count($this->all_recipients) > count($bad_rcpt)) && !$this->smtp->data($header . $body)) { - throw new Exception($this->lang('data_not_accepted'), self::STOP_CRITICAL); - } - - $smtp_transaction_id = $this->smtp->getLastTransactionID(); - - if ($this->SMTPKeepAlive) { - $this->smtp->reset(); - } else { - $this->smtp->quit(); - $this->smtp->close(); - } - - foreach ($callbacks as $cb) { - $this->doCallback( - $cb['issent'], - [[$cb['to'], $cb['name']]], - [], - [], - $this->Subject, - $body, - $this->From, - ['smtp_transaction_id' => $smtp_transaction_id] - ); - } - - //Create error message for any bad addresses - if (count($bad_rcpt) > 0) { - $errstr = ''; - foreach ($bad_rcpt as $bad) { - $errstr .= $bad['to'] . ': ' . $bad['error']; - } - throw new Exception($this->lang('recipients_failed') . $errstr, self::STOP_CONTINUE); - } - - return true; - } - - /** - * Initiate a connection to an SMTP server. - * Returns false if the operation failed. - * - * @param array $options An array of options compatible with stream_context_create() - * - * @throws Exception - * - * @uses \PHPMailer\PHPMailer\SMTP - * - * @return bool - */ - public function smtpConnect($options = null) - { - if (null === $this->smtp) { - $this->smtp = $this->getSMTPInstance(); - } - - //If no options are provided, use whatever is set in the instance - if (null === $options) { - $options = $this->SMTPOptions; - } - - //Already connected? - if ($this->smtp->connected()) { - return true; - } - - $this->smtp->setTimeout($this->Timeout); - $this->smtp->setDebugLevel($this->SMTPDebug); - $this->smtp->setDebugOutput($this->Debugoutput); - $this->smtp->setVerp($this->do_verp); - if ($this->Host === null) { - $this->Host = 'localhost'; - } - $hosts = explode(';', $this->Host); - $lastexception = null; - - foreach ($hosts as $hostentry) { - $hostinfo = []; - if ( - !preg_match( - '/^(?:(ssl|tls):\/\/)?(.+?)(?::(\d+))?$/', - trim($hostentry), - $hostinfo - ) - ) { - $this->edebug($this->lang('invalid_hostentry') . ' ' . trim($hostentry)); - //Not a valid host entry - continue; - } - //$hostinfo[1]: optional ssl or tls prefix - //$hostinfo[2]: the hostname - //$hostinfo[3]: optional port number - //The host string prefix can temporarily override the current setting for SMTPSecure - //If it's not specified, the default value is used - - //Check the host name is a valid name or IP address before trying to use it - if (!static::isValidHost($hostinfo[2])) { - $this->edebug($this->lang('invalid_host') . ' ' . $hostinfo[2]); - continue; - } - $prefix = ''; - $secure = $this->SMTPSecure; - $tls = (static::ENCRYPTION_STARTTLS === $this->SMTPSecure); - if ('ssl' === $hostinfo[1] || ('' === $hostinfo[1] && static::ENCRYPTION_SMTPS === $this->SMTPSecure)) { - $prefix = 'ssl://'; - $tls = false; //Can't have SSL and TLS at the same time - $secure = static::ENCRYPTION_SMTPS; - } elseif ('tls' === $hostinfo[1]) { - $tls = true; - //TLS doesn't use a prefix - $secure = static::ENCRYPTION_STARTTLS; - } - //Do we need the OpenSSL extension? - $sslext = defined('OPENSSL_ALGO_SHA256'); - if (static::ENCRYPTION_STARTTLS === $secure || static::ENCRYPTION_SMTPS === $secure) { - //Check for an OpenSSL constant rather than using extension_loaded, which is sometimes disabled - if (!$sslext) { - throw new Exception($this->lang('extension_missing') . 'openssl', self::STOP_CRITICAL); - } - } - $host = $hostinfo[2]; - $port = $this->Port; - if ( - array_key_exists(3, $hostinfo) && - is_numeric($hostinfo[3]) && - $hostinfo[3] > 0 && - $hostinfo[3] < 65536 - ) { - $port = (int) $hostinfo[3]; - } - if ($this->smtp->connect($prefix . $host, $port, $this->Timeout, $options)) { - try { - if ($this->Helo) { - $hello = $this->Helo; - } else { - $hello = $this->serverHostname(); - } - $this->smtp->hello($hello); - //Automatically enable TLS encryption if: - //* it's not disabled - //* we are not connecting to localhost - //* we have openssl extension - //* we are not already using SSL - //* the server offers STARTTLS - if ( - $this->SMTPAutoTLS && - $this->Host !== 'localhost' && - $sslext && - $secure !== 'ssl' && - $this->smtp->getServerExt('STARTTLS') - ) { - $tls = true; - } - if ($tls) { - if (!$this->smtp->startTLS()) { - $message = $this->getSmtpErrorMessage('connect_host'); - throw new Exception($message); - } - //We must resend EHLO after TLS negotiation - $this->smtp->hello($hello); - } - if ( - $this->SMTPAuth && !$this->smtp->authenticate( - $this->Username, - $this->Password, - $this->AuthType, - $this->oauth - ) - ) { - throw new Exception($this->lang('authenticate')); - } - - return true; - } catch (Exception $exc) { - $lastexception = $exc; - $this->edebug($exc->getMessage()); - //We must have connected, but then failed TLS or Auth, so close connection nicely - $this->smtp->quit(); - } - } - } - //If we get here, all connection attempts have failed, so close connection hard - $this->smtp->close(); - //As we've caught all exceptions, just report whatever the last one was - if ($this->exceptions && null !== $lastexception) { - throw $lastexception; - } - if ($this->exceptions) { - // no exception was thrown, likely $this->smtp->connect() failed - $message = $this->getSmtpErrorMessage('connect_host'); - throw new Exception($message); - } - - return false; - } - - /** - * Close the active SMTP session if one exists. - */ - public function smtpClose() - { - if ((null !== $this->smtp) && $this->smtp->connected()) { - $this->smtp->quit(); - $this->smtp->close(); - } - } - - /** - * Set the language for error messages. - * The default language is English. - * - * @param string $langcode ISO 639-1 2-character language code (e.g. French is "fr") - * Optionally, the language code can be enhanced with a 4-character - * script annotation and/or a 2-character country annotation. - * @param string $lang_path Path to the language file directory, with trailing separator (slash) - * Do not set this from user input! - * - * @return bool Returns true if the requested language was loaded, false otherwise. - */ - public function setLanguage($langcode = 'en', $lang_path = '') - { - //Backwards compatibility for renamed language codes - $renamed_langcodes = [ - 'br' => 'pt_br', - 'cz' => 'cs', - 'dk' => 'da', - 'no' => 'nb', - 'se' => 'sv', - 'rs' => 'sr', - 'tg' => 'tl', - 'am' => 'hy', - ]; - - if (array_key_exists($langcode, $renamed_langcodes)) { - $langcode = $renamed_langcodes[$langcode]; - } - - //Define full set of translatable strings in English - $PHPMAILER_LANG = [ - 'authenticate' => 'SMTP Error: Could not authenticate.', - 'buggy_php' => 'Your version of PHP is affected by a bug that may result in corrupted messages.' . - ' To fix it, switch to sending using SMTP, disable the mail.add_x_header option in' . - ' your php.ini, switch to MacOS or Linux, or upgrade your PHP to version 7.0.17+ or 7.1.3+.', - 'connect_host' => 'SMTP Error: Could not connect to SMTP host.', - 'data_not_accepted' => 'SMTP Error: data not accepted.', - 'empty_message' => 'Message body empty', - 'encoding' => 'Unknown encoding: ', - 'execute' => 'Could not execute: ', - 'extension_missing' => 'Extension missing: ', - 'file_access' => 'Could not access file: ', - 'file_open' => 'File Error: Could not open file: ', - 'from_failed' => 'The following From address failed: ', - 'instantiate' => 'Could not instantiate mail function.', - 'invalid_address' => 'Invalid address: ', - 'invalid_header' => 'Invalid header name or value', - 'invalid_hostentry' => 'Invalid hostentry: ', - 'invalid_host' => 'Invalid host: ', - 'mailer_not_supported' => ' mailer is not supported.', - 'provide_address' => 'You must provide at least one recipient email address.', - 'recipients_failed' => 'SMTP Error: The following recipients failed: ', - 'signing' => 'Signing Error: ', - 'smtp_code' => 'SMTP code: ', - 'smtp_code_ex' => 'Additional SMTP info: ', - 'smtp_connect_failed' => 'SMTP connect() failed.', - 'smtp_detail' => 'Detail: ', - 'smtp_error' => 'SMTP server error: ', - 'variable_set' => 'Cannot set or reset variable: ', - ]; - if (empty($lang_path)) { - //Calculate an absolute path so it can work if CWD is not here - $lang_path = dirname(__DIR__) . DIRECTORY_SEPARATOR . 'language' . DIRECTORY_SEPARATOR; - } - - //Validate $langcode - $foundlang = true; - $langcode = strtolower($langcode); - if ( - !preg_match('/^(?P[a-z]{2})(?P - - - \ No newline at end of file diff --git a/backup_2026-05-13_pre_intelligent_retrieval/app/api/__init__.py b/backup_2026-05-13_pre_intelligent_retrieval/app/api/__init__.py deleted file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..0000000000000000000000000000000000000000 diff --git a/backup_2026-05-13_pre_intelligent_retrieval/app/api/schemas.py b/backup_2026-05-13_pre_intelligent_retrieval/app/api/schemas.py deleted file mode 100644 index 3769f29cf891fb8935d2aff8e74c38c3bc4ebb2c..0000000000000000000000000000000000000000 --- a/backup_2026-05-13_pre_intelligent_retrieval/app/api/schemas.py +++ /dev/null @@ -1,27 +0,0 @@ -from typing import List, Dict, Any - -from pydantic import BaseModel, Field - - -class ChatRequest(BaseModel): - message: str = Field(..., min_length=1) - history: List[Dict[str, str]] = Field(default_factory=list) - session_id: str | None = Field(default=None, description="Unique session identifier for chat persistence") - - -class ChatResponse(BaseModel): - reply: str - retrieved_chunks: List[Dict[str, Any]] = Field(default_factory=list) - - -class HealthResponse(BaseModel): - status: str - docs_loaded: bool - index_ready: bool - chunk_count: int - index_path: str - - -class SessionUpdateNameRequest(BaseModel): - session_id: str - user_name: str diff --git a/backup_2026-05-13_pre_intelligent_retrieval/app/core/__init__.py b/backup_2026-05-13_pre_intelligent_retrieval/app/core/__init__.py deleted file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..0000000000000000000000000000000000000000 diff --git a/backup_2026-05-13_pre_intelligent_retrieval/app/core/config.py b/backup_2026-05-13_pre_intelligent_retrieval/app/core/config.py deleted file mode 100644 index bd1399ae9d701881c8cd38ffffef1e1c10cd073e..0000000000000000000000000000000000000000 --- a/backup_2026-05-13_pre_intelligent_retrieval/app/core/config.py +++ /dev/null @@ -1,57 +0,0 @@ -from functools import lru_cache -from pathlib import Path -from pydantic import Field, model_validator -from pydantic_settings import BaseSettings, SettingsConfigDict - - -class Settings(BaseSettings): - model_config = SettingsConfigDict(env_file=".env", env_file_encoding="utf-8") - - app_name: str = "Fast RAG Chatbot" - llm_provider: str = Field(default="openai", alias="LLM_PROVIDER") - groq_api_key: str = Field(default="", alias="GROQ_API_KEY") - groq_model: str = Field(default="qwen/qwen3-32b", alias="GROQ_MODEL") - groq_rewrite_model: str = Field(default="llama-3.1-8b-instant", alias="GROQ_REWRITE_MODEL") - fireworks_api_key: str = Field(default="", alias="FIREWORKS_API_KEY") - fireworks_model: str = Field(default="accounts/fireworks/models/qwen3-30b-a3b-instruct-2507", alias="FIREWORKS_MODEL") - fireworks_rewrite_model: str = Field(default="accounts/fireworks/models/qwen3-8b", alias="FIREWORKS_REWRITE_MODEL") - openai_api_key: str = Field(default="", alias="OPENAI_API_KEY") - openai_model: str = Field(default="gpt-4o-mini", alias="OPENAI_MODEL") - openai_rewrite_model: str = Field(default="gpt-4o-mini", alias="OPENAI_REWRITE_MODEL") - hf_api_key: str = Field(default="", alias="HF_API_KEY") - hf_model: str = Field(default="meta-llama/Llama-3.1-8B-Instruct", alias="HF_MODEL") - embedding_model: str = Field(default="BAAI/bge-small-en-v1.5", alias="EMBEDDING_MODEL") - reranker_model: str = Field(default="cross-encoder/ms-marco-MiniLM-L-6-v2", alias="RERANKER_MODEL") - docs_dir: Path = Field(default=Path("docs"), alias="DOCS_DIR") - @model_validator(mode='after') - def set_persistent_paths(self) -> 'Settings': - # If we are on Hugging Face (detected by /data volume), - # force use of /data for persistence regardless of other settings. - if Path("/data").exists(): - self.index_dir = Path("/data/index") - self.sessions_dir = Path("/data/sessions") - return self - - index_dir: Path = Field(default=Path("data/index"), alias="INDEX_DIR") - sessions_dir: Path = Field(default=Path("data/sessions"), alias="SESSIONS_DIR") - chunk_size_tokens: int = 350 # Reduced for TPM safety - chunk_overlap_tokens: int = 80 - top_k: int = Field(default=8, alias="TOP_K") - max_context_chunks: int = 4 # _build_context() caps at 1500 words anyway; 4 chunks is sufficient - request_timeout_s: float = 20.0 - cors_allow_origins: str = Field(default="*", alias="CORS_ALLOW_ORIGINS") - api_key: str = Field(default="", alias="API_KEY") - rate_limit_requests: int = Field(default=60, alias="RATE_LIMIT_REQUESTS") - rate_limit_window_seconds: int = Field(default=60, alias="RATE_LIMIT_WINDOW_SECONDS") - - # SMTP Settings for OTP - smtp_server: str = Field(default="smtp.gmail.com", alias="SMTP_SERVER") - smtp_port: int = Field(default=465, alias="SMTP_PORT") - smtp_user: str = Field(default="", alias="SMTP_USER") - smtp_pass: str = Field(default="", alias="SMTP_PASS") - admin_email: str = Field(default="randomjoedown@gmail.com", alias="ADMIN_EMAIL") - - -@lru_cache -def get_settings() -> Settings: - return Settings() diff --git a/backup_2026-05-13_pre_intelligent_retrieval/app/main.py b/backup_2026-05-13_pre_intelligent_retrieval/app/main.py deleted file mode 100644 index 27479a701e24b237d59b9d6a7a37a97b699d8602..0000000000000000000000000000000000000000 --- a/backup_2026-05-13_pre_intelligent_retrieval/app/main.py +++ /dev/null @@ -1,208 +0,0 @@ -import logging - -from fastapi import FastAPI, HTTPException, Header, Request -from fastapi.responses import RedirectResponse -from fastapi.middleware.cors import CORSMiddleware -from fastapi.staticfiles import StaticFiles -import os - -from app.api.schemas import ChatRequest, ChatResponse, HealthResponse, SessionUpdateNameRequest -from app.core.config import get_settings -from app.services.embeddings import EmbeddingService -from app.services.llm import LLMService -from app.services.rag_pipeline import RAGPipeline -from app.services.rate_limiter import InMemoryRateLimiter -from app.services.vector_store import FaissVectorStore -from app.services.reranker import RerankerService -from app.ui_gradio import demo -from app.admin.router import admin_router -from app.services import session_store as _ss -import asyncio -import gradio as gr - -logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(name)s - %(message)s") -logger = logging.getLogger(__name__) - -settings = get_settings() -embedding_service = EmbeddingService(settings.embedding_model) -vector_store = FaissVectorStore( - embedding_service=embedding_service, - docs_dir=settings.docs_dir, - index_dir=settings.index_dir, - chunk_size_tokens=settings.chunk_size_tokens, - chunk_overlap_tokens=settings.chunk_overlap_tokens, -) -llm_service = LLMService( - provider=settings.llm_provider, - groq_api_key=settings.groq_api_key, - groq_model=settings.groq_model, - groq_rewrite_model=settings.groq_rewrite_model, - hf_api_key=settings.hf_api_key, - hf_model=settings.hf_model, - fireworks_api_key=settings.fireworks_api_key, - fireworks_model=settings.fireworks_model, - fireworks_rewrite_model=settings.fireworks_rewrite_model, - timeout_s=settings.request_timeout_s, -) -reranker_service = RerankerService(settings.reranker_model) -pipeline = RAGPipeline( - vector_store=vector_store, - llm_service=llm_service, - reranker=reranker_service, - top_k=settings.top_k, - max_context_chunks=settings.max_context_chunks -) - -# ── FastAPI App ─────────────────────────────────────────────────────────────── -fastapi_app = FastAPI(title=settings.app_name) - -allow_origins = [o.strip() for o in settings.cors_allow_origins.split(",") if o.strip()] -fastapi_app.add_middleware( - CORSMiddleware, - allow_origins=["*"], # Explicitly allow all for testing - allow_credentials=True, - allow_methods=["*"], - allow_headers=["*"], -) - -rate_limiter = InMemoryRateLimiter(settings.rate_limit_requests, settings.rate_limit_window_seconds) - -# ── Admin Panel (mounted on FastAPI before Gradio wrapping) ─────────────────── -fastapi_app.include_router(admin_router) - -# ── Static Files ────────────────────────────────────────────────────────────── -static_dir = os.path.join(os.path.dirname(os.path.dirname(__file__)), "static") -if os.path.exists(static_dir): - fastapi_app.mount("/static", StaticFiles(directory=static_dir), name="static") - -@fastapi_app.get("/widget", include_in_schema=False) -async def get_widget(): - """Serves the chat widget HTML with explicit CORS headers.""" - from fastapi.responses import FileResponse - path = os.path.join(static_dir, "addon.html") - if not os.path.exists(path): - raise HTTPException(status_code=404, detail="Widget not found") - return FileResponse( - path, - headers={ - "Access-Control-Allow-Origin": "*", - "Cache-Control": "no-cache, no-store, must-revalidate" - } - ) - - -@fastapi_app.on_event("startup") -def startup_event() -> None: - logger.info("─── RAG SERVICE STARTUP ───") - logger.info("Provider: %s", settings.llm_provider) - logger.info("Fireworks Key: %s", "PRESENT" if settings.fireworks_api_key else "MISSING") - logger.info("Groq Key: %s", "PRESENT" if settings.groq_api_key else "MISSING") - logger.info("Index Directory: %s", settings.index_dir) - - # Verify persistence mount - if str(settings.index_dir).startswith("/data"): - if os.path.exists("/data"): - logger.info("✅ Persistent storage /data detected and in use.") - else: - logger.warning("⚠️ /data requested but NOT found on filesystem!") - else: - logger.warning("⚠️ Using ephemeral storage (data will be lost on rebuild).") - - logger.info("Loading/building FAISS index...") - vector_store.build_or_load() - logger.info("RAG service started with %s chunks", len(vector_store.metadata)) - - -@fastapi_app.on_event("shutdown") -async def shutdown_event() -> None: - """Gracefully close the shared httpx client to free connections.""" - logger.info("─── RAG SERVICE SHUTDOWN — closing HTTP client ───") - await llm_service.close() - - -@fastapi_app.get("/", include_in_schema=False) -async def root(): - """Prevent direct browser access to the root URL.""" - raise HTTPException( - status_code=403, - detail="Direct access to this resource is restricted. Please use the authorized interface." - ) - - -@fastapi_app.get("/health", response_model=HealthResponse) -def health() -> HealthResponse: - h = vector_store.health() - return HealthResponse(status="ok", **h) - - -@fastapi_app.post("/api/chat", response_model=ChatResponse) -async def chat( - payload: ChatRequest, - request: Request, - x_api_key: str | None = Header(default=None), -) -> ChatResponse: - if not payload.message.strip(): - raise HTTPException(status_code=400, detail="message must not be empty") - if settings.api_key and x_api_key != settings.api_key: - raise HTTPException(status_code=401, detail="unauthorized") - - client_ip = request.client.host if request.client else "unknown" - if not rate_limiter.allow(client_ip): - raise HTTPException(status_code=429, detail="rate limit exceeded") - - logger.info("Incoming chat request: msg='%s', sid='%s'", payload.message[:30], payload.session_id) - - # Fetch user_name if available for personalization - user_name = None - if payload.session_id: - sess_data = await _ss.get_session(payload.session_id) - user_name = sess_data.get("user_name") - - try: - result = await pipeline.chat(payload.message, payload.history, user_name=user_name) - - # ── Fire-and-forget: persist to session store in background ── - if payload.session_id: - asyncio.create_task(_ss.save_message(payload.session_id, payload.message, result["reply"])) - - return ChatResponse(**result) - except Exception as e: - import httpx - import traceback - logger.error(f"Error in chat endpoint: {str(e)}\n{traceback.format_exc()}") - - error_msg = "⚠️ Oops! Something went wrong." - if isinstance(e, httpx.HTTPStatusError): - if e.response.status_code == 429: - error_msg = "⚠️ Rate limit reached. Please slow down a bit!" - else: - error_msg = "⚠️ I encountered an error. Please try again in a few seconds." - - return ChatResponse(reply=error_msg, retrieved_chunks=[]) - - -@fastapi_app.post("/api/session/update-name") -async def update_session_user_name(payload: SessionUpdateNameRequest): - """Updates the user name and renames the session file to match the name.""" - new_id = await _ss.rename_session(payload.session_id, payload.user_name) - return {"status": "ok", "new_session_id": new_id} - - -@fastapi_app.get("/api/session/history/{session_id}") -async def get_session_history(session_id: str): - """Returns the chat history for a session formatted for the frontend.""" - data = await _ss.get_session(session_id) - if not data: - return {"history": []} - - history = [] - for msg in data.get("messages", []): - history.append({"role": "user", "content": msg["question"]}) - history.append({"role": "assistant", "content": msg["answer"]}) - return {"history": history, "user_name": data.get("user_name", "")} - - -# ── Mount Gradio at /ui — MUST be last, wraps the FastAPI app ───────────────── -# All FastAPI routes (including /admin) are already registered above and are -# preserved inside the Gradio-wrapped Starlette app. -app = gr.mount_gradio_app(fastapi_app, demo, path="/ui") diff --git a/backup_2026-05-13_pre_intelligent_retrieval/app/services/__init__.py b/backup_2026-05-13_pre_intelligent_retrieval/app/services/__init__.py deleted file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..0000000000000000000000000000000000000000 diff --git a/backup_2026-05-13_pre_intelligent_retrieval/app/services/chunker.py b/backup_2026-05-13_pre_intelligent_retrieval/app/services/chunker.py deleted file mode 100644 index d033e1f5f243452dae87fb2a556d8d9e593b854f..0000000000000000000000000000000000000000 --- a/backup_2026-05-13_pre_intelligent_retrieval/app/services/chunker.py +++ /dev/null @@ -1,39 +0,0 @@ -from typing import List, Dict - - -def _tokenize(text: str) -> List[str]: - return text.split() - - -def chunk_documents( - documents: List[Dict[str, str]], - chunk_size_tokens: int = 420, - chunk_overlap_tokens: int = 80, -) -> List[Dict[str, str]]: - chunks: List[Dict[str, str]] = [] - step = max(1, chunk_size_tokens - chunk_overlap_tokens) - - for doc in documents: - tokens = _tokenize(doc["text"]) - if not tokens: - continue - - i = 0 - chunk_id = 0 - while i < len(tokens): - window = tokens[i : i + chunk_size_tokens] - if not window: - break - - chunk_text = " ".join(window) - chunks.append( - { - "id": f"{doc['source']}::chunk_{chunk_id}", - "source": doc["source"], - "text": chunk_text, - } - ) - chunk_id += 1 - i += step - - return chunks diff --git a/backup_2026-05-13_pre_intelligent_retrieval/app/services/document_loader.py b/backup_2026-05-13_pre_intelligent_retrieval/app/services/document_loader.py deleted file mode 100644 index 4741d73b2686c554ff22e184687ff7cf12e718c7..0000000000000000000000000000000000000000 --- a/backup_2026-05-13_pre_intelligent_retrieval/app/services/document_loader.py +++ /dev/null @@ -1,43 +0,0 @@ -import logging -from pathlib import Path -from typing import List - -from pypdf import PdfReader - -logger = logging.getLogger(__name__) - -def _clean_text(text: str) -> str: - lines = [line.strip() for line in text.splitlines() if line.strip()] - merged = " ".join(lines) - return " ".join(merged.split()) - - -def load_documents(docs_dir: Path) -> List[dict]: - docs: List[dict] = [] - if not docs_dir.exists(): - return docs - - for file_path in sorted(docs_dir.rglob("*")): - if not file_path.is_file(): - continue - - try: - suffix = file_path.suffix.lower() - if suffix == ".txt": - raw = file_path.read_text(encoding="utf-8", errors="ignore") - cleaned = _clean_text(raw) - if cleaned: - docs.append({"source": str(file_path), "text": cleaned}) - elif suffix == ".pdf": - reader = PdfReader(str(file_path)) - pages = [] - for page in reader.pages: - pages.append(page.extract_text() or "") - cleaned = _clean_text("\n".join(pages)) - if cleaned: - docs.append({"source": str(file_path), "text": cleaned}) - except Exception as e: - logger.error(f"Error loading document {file_path}: {e}") - continue - - return docs diff --git a/backup_2026-05-13_pre_intelligent_retrieval/app/services/embeddings.py b/backup_2026-05-13_pre_intelligent_retrieval/app/services/embeddings.py deleted file mode 100644 index a920c0ffb22715a7eeefe8790c45ebb7b67cdaaf..0000000000000000000000000000000000000000 --- a/backup_2026-05-13_pre_intelligent_retrieval/app/services/embeddings.py +++ /dev/null @@ -1,34 +0,0 @@ -from typing import List -from functools import lru_cache - -import numpy as np -from sentence_transformers import SentenceTransformer - - -class EmbeddingService: - def __init__(self, model_name: str) -> None: - self.model = SentenceTransformer(model_name) - - def encode(self, texts: List[str]) -> np.ndarray: - vectors = self.model.encode( - texts, - normalize_embeddings=True, - show_progress_bar=False, - convert_to_numpy=True, - ) - return np.asarray(vectors, dtype=np.float32) - - @lru_cache(maxsize=2048) - def encode_query_cached(self, text: str) -> bytes: - vec = self.model.encode( - [text], - normalize_embeddings=True, - show_progress_bar=False, - convert_to_numpy=True, - ) - arr = np.asarray(vec, dtype=np.float32) - return arr.tobytes() - - def encode_query(self, text: str) -> np.ndarray: - buf = self.encode_query_cached(text) - return np.frombuffer(buf, dtype=np.float32).reshape(1, -1) diff --git a/backup_2026-05-13_pre_intelligent_retrieval/app/services/llm.py b/backup_2026-05-13_pre_intelligent_retrieval/app/services/llm.py deleted file mode 100644 index 11d036a73d452928b5cfafcea447135b0ede84b8..0000000000000000000000000000000000000000 --- a/backup_2026-05-13_pre_intelligent_retrieval/app/services/llm.py +++ /dev/null @@ -1,378 +0,0 @@ -from typing import List, Dict -import httpx -import re -import logging - -_log = logging.getLogger(__name__) - -# ═══════════════════════════════════════════════════════════════════════ -# MASTER SYSTEM PROMPT — Martechsol HR Assistant -# Intelligence: Understand Intent → Retrieve Facts → Respond Precisely -# ═══════════════════════════════════════════════════════════════════════ -SYSTEM_PROMPT = """You are the Martechsol HR Assistant — intelligent, precise, and formal. - -━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ -STEP 1 — UNDERSTAND THE INTENT -━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ -Read the question carefully. Identify the SINGLE core topic being asked. Apply intelligent defaults: -• "timing" / "timings" (no context) → office working hours ONLY — not payment or any other timing -• "leaves" / "leave" (no context) → leave names + day counts ONLY — NOT leave policies or eligibility -• "paid leaves" / "all leaves" → enumerate EVERY leave type with its name and count -• "salary" / "pay" (no context) → salary structure or amount — NOT payment date unless explicitly asked -• "benefits" / "perks" / "allowances" → list EVERY benefit with its name and value -• "terminate" / "termination" → resignation/termination procedure — NOT general policies -If a question has an obvious workplace context, always default to the most common interpretation. - -━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ -STEP 2 — STRICT SCOPE DISCIPLINE -━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ -Answer ONLY what was asked. NEVER expand into: -• Policies, approval processes, eligibility rules, or consequences — unless user asks for policy/process -• Related topics the user did not mention -• Broad overviews when a specific fact was requested -• Context that wasn't in the question - -━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ -STEP 3 — FORMAT DECISION TABLE (MANDATORY — follow exactly) -━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ - -Use this table to pick the format. Do NOT deviate. - - QUESTION TYPE → FORMAT TO USE - ───────────────────────────────────────────────────── - "how many X leaves?" → FORMAT A (one number only) - "what is X leave?" → FORMAT A (one sentence, count + 1 key fact) - "how to apply / how to get X leave" → FORMAT C (procedure for THAT leave ONLY) - "what are all leaves / list all X" → FORMAT B (full exhaustive list) - "paid leaves / all paid leaves" → FORMAT B (full exhaustive list) - "what is the policy for X?" → FORMAT C (policy for THAT leave ONLY) - "and X?" (follow-up in conversation) → FORMAT A (answer only the new X, not a full list) - -FORMAT A — SINGLE FACT - Rule: ONE complete sentence. Maximum 25–30 words. Never cut mid-sentence. - Example: You are entitled to 8 Sick Leave days per year. - -FORMAT B — EXHAUSTIVE LIST - Trigger: ONLY when user says "all leaves", "all benefits", "list all X", "paid leaves", "what leaves". - Rule: - • Include EVERY single item found — omitting even one is FORBIDDEN - • One item per line: Item Name: value
- • No intro sentence, no closing sentence, no extra commentary - Example: - Casual Leave: 10 days
- Sick Leave: 8 days
- Annual Leave: 14 days
- Maternity Leave: 90 days
- Paternity Leave: 3 days
- Hajj Leave: 30 days
- ...(list every item — do NOT stop early) - -FORMAT C — BRIEF EXPLANATION (procedure / how-to) - Trigger: ONLY when user asks "how to apply", "how to get", "what is the process", "how does X work". - Rule: - • Answer ONLY for the SPECIFIC leave type asked — do NOT list all leaves - • Maximum 3 bullet points - • Each bullet = one complete, factual sentence. No filler words. - Example for "how to get sick leave": - • Notify your supervisor or HR within 2 hours of your shift start if absent due to illness. - • Submit your leave application immediately upon returning to work. - • Provide a medical certificate; failure to do so converts the leave to unpaid. - -━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ -STRICT QUALITY RULES -━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ -✓ ZERO hallucination — every fact must exist in Expert Data only. No guessing. -✓ If not in Expert Data → reply exactly: "I don't have that information." -✓ Never cut a sentence mid-way — always complete every sentence fully -✓ NEVER mention: "document", "handbook", "manual", "policy file", or any source reference -✓ Use bold for names, numbers, dates, leave types, and all key terms -✓ Use
between list items for clean vertical spacing -✓ Tone: formal, warm, and professional — never robotic, never chatty -✓ Do NOT add greetings, closings, or "Is there anything else?" type phrases""" - - - -def _build_context(chunks: List[Dict[str, str]], max_words: int = 1500) -> str: - """Combines retrieved chunks into a clean context string, capped at max_words. - Prevents TPM spikes when chunks are unexpectedly large.""" - if not chunks: - return "" - parts = [] - total_words = 0 - for c in chunks: - text = c['text'] - word_count = len(text.split()) - if total_words + word_count > max_words: - # Add a trimmed version of this chunk if possible - remaining = max_words - total_words - if remaining > 30: # Only worth adding if meaningful content remains - trimmed_words = text.split()[:remaining] - parts.append(" ".join(trimmed_words)) - break - parts.append(text) - total_words += word_count - return "\n\n".join(parts) - - -class LLMService: - def __init__( - self, - provider: str, - groq_api_key: str, - groq_model: str, - groq_rewrite_model: str, - hf_api_key: str, - hf_model: str, - fireworks_api_key: str = "", - fireworks_model: str = "accounts/fireworks/models/qwen3-32b", - fireworks_rewrite_model: str = "accounts/fireworks/models/llama-v3p1-8b-instruct", - timeout_s: float = 20.0, - ) -> None: - self.provider = provider.lower().strip() - self.groq_api_key = groq_api_key - self.groq_model = groq_model - self.groq_rewrite_model = groq_rewrite_model - self.hf_api_key = hf_api_key - self.hf_model = hf_model - self.fireworks_api_key = fireworks_api_key - self.fireworks_model = fireworks_model - self.fireworks_rewrite_model = fireworks_rewrite_model - self.timeout_s = timeout_s - # Persistent client — reused across all calls; eliminates TCP+TLS handshake per request - self._client = httpx.AsyncClient( - timeout=httpx.Timeout(timeout_s), - limits=httpx.Limits(max_keepalive_connections=5, max_connections=10), - ) - - async def close(self) -> None: - """Cleanly closes the shared HTTP client. Call on app shutdown.""" - await self._client.aclose() - - async def generate_multi_queries(self, query: str, history: List[Dict[str, str]]) -> List[str]: - """Generates multiple search queries to capture broader context from the document.""" - prompt = f"""You are an intelligent HR search optimizer. A user asked: "{query}" - -STEP 1 — UNDERSTAND THE INTENT: -What is the user ACTUALLY asking about? Apply workplace common sense: -- "timing" / "timings" (no qualifier) = office working hours, workday schedule -- "leaves" / "paid leaves" / "all leaves" = all leave types available to employees -- "salary" / "pay" = salary structure or amount -- "benefits" / "allowances" = employee benefits and perks -- "terminate" / "resign" = termination or resignation process - -STEP 2 — GENERATE 3 TARGETED SEARCH QUERIES: -Write 3 diverse search queries to retrieve ALL relevant information from an employee handbook. -For the identified intent, cover synonyms, related terms, and sub-categories: -- For office timings → search: working hours, office schedule, workday timing, shift hours -- For leaves → Query 1 MUST be a keyword dump: "casual leave sick leave annual leave maternity paternity hajj bereavement study unauthorized" -- For salary → cover: salary structure, payroll, compensation, increments, deductions -- For benefits → cover: allowances, perks, medical, bonuses, reimbursements - -Output ONLY 3 queries, one per line. No numbers, no bullets, no explanations. - -Queries:""" - - try: - if self.provider == "fireworks": - resp = await self._call_fireworks( - prompt, [], - "You output only search queries, one per line. No explanations, no numbering.", - model_override=self.fireworks_rewrite_model - ) - else: - resp = await self._call_groq( - prompt, [], - "You output only search queries, one per line. No explanations, no numbering.", - model_override=self.groq_rewrite_model - ) - queries = [ - q.strip().lstrip('0123456789.-) ').strip() - for q in resp.split("\n") - if q.strip() and len(q.strip()) > 3 - ] - # Always include original query - if query not in queries: - queries.append(query) - return queries[:3] - except Exception: - return [query] - - async def answer( - self, - question: str, - chunks: List[Dict[str, str]], - history: List[Dict[str, str]], - user_name: str = None - ) -> str: - # Keep only last 4 messages (2 turns) for TPM safety - pruned_history = history[-4:] - context = _build_context(chunks) - - # Build system message - system_msg = SYSTEM_PROMPT - if user_name and user_name not in ("default name", "", None): - system_msg += f"\n\nYou are speaking with {user_name}." - - # Build user prompt — clean and direct - if not context: - user_prompt = question - else: - user_prompt = f"Expert Data:\n{context}\n\nQuestion: {question}" - - if self.provider == "fireworks": - return await self._call_fireworks(user_prompt, pruned_history, system_msg) - return await self._call_groq(user_prompt, pruned_history, system_msg) - - async def _call_groq( - self, - user_prompt: str, - history: List[Dict[str, str]], - system_msg: str, - model_override: str = None - ) -> str: - if not self.groq_api_key: - return "Expert access required." - - url = "https://api.groq.com/openai/v1/chat/completions" - headers = { - "Authorization": f"Bearer {self.groq_api_key}", - "Content-Type": "application/json" - } - - messages = [{"role": "system", "content": system_msg}] - for msg in history: - messages.append({"role": msg["role"], "content": msg["content"]}) - messages.append({"role": "user", "content": user_prompt}) - - target_model = model_override or self.groq_model - - # Qwen3 and DeepSeek-R1 are thinking models - is_thinking_model = any(k in target_model.lower() for k in ["deepseek", "qwen3", "qwen/qwen3"]) - temp = 0.6 if is_thinking_model else 0.0 - - # For Qwen3: suppress thinking mode on answer calls to reduce latency & TPM usage. - # /no_think is Qwen3's built-in signal to skip chain-of-thought reasoning. - # We ONLY suppress for the main answer model, not the rewrite model. - if is_thinking_model and model_override is None: - messages[-1]["content"] = "/no_think\n\n" + messages[-1]["content"] - - payload = { - "model": target_model, - "temperature": temp, - "max_tokens": 512, # Enough for complete listings with HTML, well within TPM limits - "messages": messages, - } - - resp = await self._client.post(url, headers=headers, json=payload) - if resp.status_code >= 400: - _log.error("Groq API Error %s: %s", resp.status_code, resp.text) - resp.raise_for_status() - data = resp.json() - content = data["choices"][0]["message"]["content"].strip() - - # ── Post-Processing: Strip all internal reasoning artifacts ── - - # 1. Strip ... blocks (Qwen3, DeepSeek-R1) - content = re.sub(r'.*?', '', content, flags=re.DOTALL).strip() - - # 2. Strip leading conversational filler (single line only, not entire content) - content = re.sub( - r'^(Okay[,.]?\s*|Alright[,.]?\s*|Sure[,.]?\s*|Let\'s see[,.]?\s*|' - r'Based on (?:the )?(?:provided |available )?(?:data|information|context)[,.]?\s*|' - r'According to (?:the )?(?:provided |available )?(?:data|information)[,.]?\s*)', - '', content, flags=re.IGNORECASE - ).strip() - - # 3. Remove lines that are pure internal self-talk (only if they appear alone at start) - lines = content.split('\n') - filtered = [] - for i, line in enumerate(lines): - is_self_talk = bool(re.match( - r'^\s*(I need to|I will|I should|I\'m going to|Let me|Now I|First,? I|' - r'I\'ll|The user is asking|The question is about)', - line, re.IGNORECASE - )) - if not is_self_talk: - filtered.append(line) - content = '\n'.join(filtered).strip() - - return content - - async def _call_fireworks( - self, - user_prompt: str, - history: List[Dict[str, str]], - system_msg: str, - model_override: str = None - ) -> str: - if not self.fireworks_api_key: - return "Expert access required." - - url = "https://api.fireworks.ai/inference/v1/chat/completions" - headers = { - "Authorization": f"Bearer {self.fireworks_api_key}", - "Content-Type": "application/json" - } - - messages = [{"role": "system", "content": system_msg}] - for msg in history: - messages.append({"role": msg["role"], "content": msg["content"]}) - messages.append({"role": "user", "content": user_prompt}) - - target_model = model_override or self.fireworks_model - - # Only flag ACTUAL thinking/reasoning models — NOT generic instruct models. - # e.g. qwen3-vl-30b-a3b-instruct is NOT a thinking model (no chain-of-thought). - # e.g. qwen3-vl-30b-a3b-thinking IS a thinking model. - is_thinking_model = any(k in target_model.lower() for k in [ - "-thinking", "qwq", "deepseek-r1", "qwen/qwen3" - ]) - # Use low temp for determinism on instruct models; 0.6 for thinking models - temp = 0.6 if is_thinking_model else 0.1 - - # Suppress chain-of-thought ONLY for actual thinking models on main answer calls - if is_thinking_model and model_override is None: - messages[-1]["content"] = "/no_think\n\n" + messages[-1]["content"] - - payload = { - "model": target_model, - "temperature": temp, - "max_tokens": 512, # Enough for complete listings with HTML, well within limits - "messages": messages, - } - - resp = await self._client.post(url, headers=headers, json=payload) - if resp.status_code >= 400: - error_detail = resp.text[:300] - _log.error("Fireworks API Error %s: %s", resp.status_code, resp.text) - return f"[Fireworks Error {resp.status_code}]: {error_detail}" - data = resp.json() - content = data["choices"][0]["message"]["content"].strip() - - # ── Post-Processing: identical pipeline as Groq path ── - - # 1. Strip ... blocks (Qwen3) - content = re.sub(r'.*?', '', content, flags=re.DOTALL).strip() - - # 2. Strip leading conversational filler - content = re.sub( - r'^(Okay[,.]?\s*|Alright[,.]?\s*|Sure[,.]?\s*|Let\'s see[,.]?\s*|' - r'Based on (?:the )?(?:provided |available )?(?:data|information|context)[,.]?\s*|' - r'According to (?:the )?(?:provided |available )?(?:data|information)[,.]?\s*)', - '', content, flags=re.IGNORECASE - ).strip() - - # 3. Remove lines that are pure internal self-talk - lines = content.split('\n') - filtered = [] - for line in lines: - is_self_talk = bool(re.match( - r'^\s*(I need to|I will|I should|I\'m going to|Let me|Now I|First,? I|' - r'I\'ll|The user is asking|The question is about)', - line, re.IGNORECASE - )) - if not is_self_talk: - filtered.append(line) - content = '\n'.join(filtered).strip() - - return content diff --git a/backup_2026-05-13_pre_intelligent_retrieval/app/services/rag_pipeline.py b/backup_2026-05-13_pre_intelligent_retrieval/app/services/rag_pipeline.py deleted file mode 100644 index 2c8f3a311931c4e65419f809dca1f3996582d25b..0000000000000000000000000000000000000000 --- a/backup_2026-05-13_pre_intelligent_retrieval/app/services/rag_pipeline.py +++ /dev/null @@ -1,203 +0,0 @@ -import logging -import hashlib -from collections import OrderedDict -from typing import Dict, List - -from app.services.llm import LLMService -from app.services.vector_store import FaissVectorStore -from app.services.reranker import RerankerService - -logger = logging.getLogger(__name__) - -# ── In-memory LRU answer cache ────────────────────────────────────────────── -# Keyed on MD5 of the normalized message. Holds up to 128 unique answers. -# Clears automatically on app restart (intentional — KB could be re-indexed). -_CACHE_MAX = 128 -_answer_cache: OrderedDict = OrderedDict() - - -def _cache_key(message: str) -> str: - """Returns a stable hash key for a normalized message string.""" - return hashlib.md5(message.lower().strip().encode()).hexdigest() - -# ── Local intent-based query expander ──────────────────────────────────────── -# Replaces the async Groq API call (llama-3.1-8b-instant) with deterministic -# Python rules — runs in microseconds, saves 3–6s per request. -# Rules mirror the exact intent-detection logic from the old LLM prompt. -# IMPORTANT: ordered most-specific → least-specific to prevent false matches. -_INTENT_MAP: List[tuple] = [ - # ── Leave types (most specific first) ── - ("paid leave", ["casual leave sick leave annual leave maternity paternity hajj bereavement study unauthorized", - "paid leave types employee entitlement days count"]), - ("all leave", ["casual leave sick leave annual leave maternity paternity hajj bereavement study unauthorized", - "all leave types employee entitlement days count"]), - ("maternity", ["maternity leave days duration policy", "paid leave maternity employee"]), - ("paternity", ["paternity leave days duration policy", "paid leave paternity employee"]), - ("hajj", ["hajj leave days duration policy", "paid leave hajj religious"]), - ("bereavement", ["bereavement leave death family days", "compassionate leave policy"]), - ("sick leave", ["sick leave days count policy", "medical leave employee entitlement"]), - ("casual leave", ["casual leave days count policy", "leave types employee"]), - ("annual leave", ["annual leave days count policy", "leave entitlement per year"]), - ("study leave", ["study leave education policy days", "employee study leave entitlement"]), - ("leave", ["casual leave sick leave annual leave maternity paternity hajj bereavement study unauthorized", - "leave types employee entitlement days"]), - # ── Office timing ── - ("office hour", ["office working hours schedule", "workday start end time shift"]), - ("work hour", ["office working hours schedule", "workday start end time shift"]), - ("timing", ["office working hours schedule", "workday start end time shift hours"]), - ("schedule", ["office working hours schedule", "workday start end time"]), - # ── Salary / Pay ── - ("salary", ["salary structure payroll compensation amount", "monthly pay increment deduction"]), - ("pay", ["salary structure payroll compensation", "payment date schedule"]), - ("payroll", ["salary payroll structure compensation", "monthly pay deduction"]), - ("compensation", ["salary compensation structure payroll", "monthly pay amount"]), - # ── Benefits / Allowances ── - ("allowance", ["allowances perks medical bonuses fuel transport reimbursements", "employee benefits"]), - ("benefit", ["allowances perks medical bonuses reimbursements", "employee benefits privileges"]), - ("perk", ["employee perks benefits extras privileges", "allowances bonuses"]), - ("fuel", ["fuel allowance transport reimbursement conveyance petrol"]), - ("medical", ["medical allowance health insurance coverage", "medical benefits employee"]), - ("transport", ["transport allowance fuel reimbursement conveyance"]), - ("bonus", ["bonus performance incentive annual eid festival reward"]), - # ── Termination / Resignation ── - ("terminat", ["termination resignation procedure process steps", "notice period exit policy"]), - ("resign", ["resignation procedure steps notice period", "termination exit process"]), - ("notice period", ["notice period resignation termination duration days"]), - # ── Other HR topics ── - ("probat", ["probation period duration conditions employee"]), - ("overtime", ["overtime compensation extra hours payment policy"]), - ("increment", ["salary increment raise annual review appraisal performance"]), - ("appraisal", ["performance appraisal review increment salary raise"]), - ("attendance", ["attendance policy punctuality late arrival absenteeism"]), - ("dress code", ["dress code uniform attire professional clothing policy"]), - ("remote", ["remote work work from home WFH policy telecommute"]), - ("grievance", ["grievance complaint procedure policy employee rights"]), - ("discipline", ["disciplinary action policy procedure employee"]), - ("code of conduct", ["code of conduct policy employee behaviour rules"]), -] - - -def _expand_query_locally(message: str) -> List[str]: - """ - Expands a user query into targeted search strings using keyword rules. - Replaces the async LLM rewrite call — runs in microseconds, zero API cost. - Mirrors the exact intent-detection logic previously in the llama-3.1-8b prompt. - """ - msg_lower = message.lower() - queries: List[str] = [message] # Original query always first - - for keyword, variants in _INTENT_MAP: - if keyword in msg_lower: - for v in variants: - if v not in queries: - queries.append(v) - break # Only apply first (most specific) matching intent - - return queries[:3] # Cap at 3, consistent with previous LLM behaviour - - -# RRF scores are small (e.g. 0.016–0.033), so threshold must be very low -RELEVANCE_THRESHOLD = 0.01 -# Cross-encoder logit > 0 means > 50% relevance probability -RERANK_THRESHOLD = 0.0 -# If ALL chunks fail rerank threshold, fall back to this many top chunks -RERANK_FALLBACK_N = 2 - - -class RAGPipeline: - def __init__( - self, - vector_store: FaissVectorStore, - llm_service: LLMService, - reranker: RerankerService, - top_k: int, - max_context_chunks: int - ) -> None: - self.vector_store = vector_store - self.llm_service = llm_service - self.reranker = reranker - self.top_k = top_k - self.max_context_chunks = max_context_chunks - - async def chat(self, message: str, history: List[Dict[str, str]], user_name: str = None) -> Dict[str, object]: - # ── Cache check: return instantly for repeated identical questions ── - key = _cache_key(message) - if key in _answer_cache: - _answer_cache.move_to_end(key) # Mark as recently used - logger.info("Cache HIT for: '%s'", message[:40]) - return _answer_cache[key] - - # ── Step 1: Expand query locally (no API call — instant) ── - queries = _expand_query_locally(message) - logger.info("Expanded %d queries for: '%s' → %s", len(queries), message[:40], queries) - - # ── Step 2: Collect unique chunks across all queries ── - seen_ids = set() - all_retrieved = [] - - for q in queries: - query_chunks = self.vector_store.search(q, top_k=self.top_k) - for chunk in query_chunks: - if chunk["id"] not in seen_ids: - all_retrieved.append(chunk) - seen_ids.add(chunk["id"]) - - # ── Step 3: Initial relevance filter ── - initial_chunks = [c for c in all_retrieved if c["score"] >= RELEVANCE_THRESHOLD] - - if not initial_chunks: - logger.info("No relevant chunks found for: '%s' — returning no-info response", message) - # Pass empty chunks; LLM is instructed to say "I don't have that information" - reply = await self.llm_service.answer( - question=message, - chunks=[], - history=history, - user_name=user_name - ) - return {"reply": reply, "retrieved_chunks": []} - - # ── Step 4: Deep reranking via Cross-Encoder ── - # Enrich the reranker query with the first expanded variant (queries[1]) - # to give the cross-encoder broader semantic context. - rerank_query = message - if len(queries) > 1: - rerank_query = f"{message} {queries[1]}" - - reranked_chunks = self.reranker.rerank(rerank_query, initial_chunks, top_n=self.max_context_chunks) - - # Filter by rerank score threshold - final_chunks = [c for c in reranked_chunks if c.get("rerank_score", 0) > RERANK_THRESHOLD] - - # ── Smart Fallback: if ALL chunks fail the threshold, use the top N anyway ── - # This prevents the bot from saying "I don't have info" when content WAS retrieved - # but the cross-encoder wasn't confident enough (e.g. paraphrased queries) - if not final_chunks and reranked_chunks: - final_chunks = reranked_chunks[:RERANK_FALLBACK_N] - logger.info( - "Rerank fallback activated — all chunks below threshold (best score=%.3f), using top %d", - reranked_chunks[0].get("rerank_score", 0), - len(final_chunks) - ) - - logger.info( - "Pipeline: retrieved=%d → relevance_filtered=%d → reranked=%d → final=%d (best_score=%.3f)", - len(all_retrieved), len(initial_chunks), len(reranked_chunks), len(final_chunks), - reranked_chunks[0].get("rerank_score", 0) if reranked_chunks else 0.0 - ) - - # ── Step 5: Generate answer with top-ranked context ── - reply = await self.llm_service.answer( - question=message, - chunks=final_chunks, - history=history, - user_name=user_name - ) - result = {"reply": reply, "retrieved_chunks": final_chunks} - - # ── Populate cache (evict oldest if at capacity) ── - _answer_cache[key] = result - if len(_answer_cache) > _CACHE_MAX: - _answer_cache.popitem(last=False) # Remove least-recently-used - logger.info("Cache MISS — stored answer for: '%s'", message[:40]) - - return result diff --git a/backup_2026-05-13_pre_intelligent_retrieval/app/services/rate_limiter.py b/backup_2026-05-13_pre_intelligent_retrieval/app/services/rate_limiter.py deleted file mode 100644 index 013800657d9cdbd132919274e6fae6076345bd5a..0000000000000000000000000000000000000000 --- a/backup_2026-05-13_pre_intelligent_retrieval/app/services/rate_limiter.py +++ /dev/null @@ -1,24 +0,0 @@ -import time -from collections import defaultdict, deque -from typing import Deque, Dict - - -class InMemoryRateLimiter: - def __init__(self, max_requests: int, window_seconds: int) -> None: - self.max_requests = max_requests - self.window_seconds = window_seconds - self._hits: Dict[str, Deque[float]] = defaultdict(deque) - - def allow(self, key: str) -> bool: - now = time.time() - window_start = now - self.window_seconds - q = self._hits[key] - - while q and q[0] < window_start: - q.popleft() - - if len(q) >= self.max_requests: - return False - - q.append(now) - return True diff --git a/backup_2026-05-13_pre_intelligent_retrieval/app/services/reranker.py b/backup_2026-05-13_pre_intelligent_retrieval/app/services/reranker.py deleted file mode 100644 index c7355c45b8fbe66cd53f8303744fdc8ffbac7969..0000000000000000000000000000000000000000 --- a/backup_2026-05-13_pre_intelligent_retrieval/app/services/reranker.py +++ /dev/null @@ -1,26 +0,0 @@ -from typing import List, Dict -import numpy as np -from sentence_transformers import CrossEncoder - -class RerankerService: - def __init__(self, model_name: str) -> None: - self.model = CrossEncoder(model_name) - - def rerank(self, query: str, chunks: List[Dict[str, str]], top_n: int = 4) -> List[Dict[str, str]]: - if not chunks: - return [] - - # Prepare pairs for cross-encoder - pairs = [[query, chunk["text"]] for chunk in chunks] - - # Get scores - scores = self.model.predict(pairs) - - # Add scores to chunks and sort - for i, chunk in enumerate(chunks): - chunk["rerank_score"] = float(scores[i]) - - # Sort by rerank_score descending - sorted_chunks = sorted(chunks, key=lambda x: x["rerank_score"], reverse=True) - - return sorted_chunks[:top_n] diff --git a/backup_2026-05-13_pre_intelligent_retrieval/app/services/session_store.py b/backup_2026-05-13_pre_intelligent_retrieval/app/services/session_store.py deleted file mode 100644 index 7eebb462c8d16ffb5330ec9d6cde539624bdb8cf..0000000000000000000000000000000000000000 --- a/backup_2026-05-13_pre_intelligent_retrieval/app/services/session_store.py +++ /dev/null @@ -1,303 +0,0 @@ -""" -session_store.py -──────────────── -Async, fire-and-forget JSON-based chat session storage. -Each browser session gets its own JSON file in data/sessions/. - -Rules: - • NEVER raises exceptions that propagate upward — any failure is silently logged. - • ALL disk I/O is done with asyncio to avoid blocking the event loop. - • Thread-safe: uses an asyncio.Lock per session_id to avoid race conditions. -""" - -import asyncio -import json -import logging -import os -from datetime import datetime, timezone -from pathlib import Path -from typing import Dict - -from app.core.config import get_settings -logger = logging.getLogger(__name__) -settings = get_settings() - -# ── Directory where session JSON files are stored ─────────────────────────── -def _get_sessions_dir() -> Path: - """Returns the current sessions directory from settings.""" - return settings.sessions_dir - - -# ── Per-session locks to prevent concurrent write corruption ───────────────── -_session_locks: Dict[str, asyncio.Lock] = {} -_locks_meta_lock = asyncio.Lock() # protects the _session_locks dict itself - - -def _get_session_path(session_id: str) -> Path: - """Returns the absolute path for a session's JSON file.""" - return _get_sessions_dir() / f"{session_id}.json" - - -def _now_iso() -> str: - """Returns the current UTC time as an ISO-8601 string.""" - return datetime.now(timezone.utc).isoformat() - - -async def _get_lock(session_id: str) -> asyncio.Lock: - """Gets or creates a per-session asyncio.Lock in a thread-safe manner.""" - async with _locks_meta_lock: - if session_id not in _session_locks: - _session_locks[session_id] = asyncio.Lock() - return _session_locks[session_id] - - -def _ensure_sessions_dir() -> None: - """Creates the sessions directory if it does not exist. Sync, called once.""" - try: - _get_sessions_dir().mkdir(parents=True, exist_ok=True) - except OSError as exc: - logger.error("session_store: cannot create sessions dir: %s", exc) - - -# Call once at import time so the directory is always ready. -_ensure_sessions_dir() - - -async def ensure_session(session_id: str) -> None: - """ - Creates the session JSON file if it doesn't exist yet. - Safe to call on every page load — does nothing if file exists. - """ - path = _get_session_path(session_id) - if path.exists(): - return - lock = await _get_lock(session_id) - async with lock: - # Double-check after acquiring lock - if path.exists(): - return - skeleton = { - "session_id": session_id, - "user_name": "default name", - "created_at": _now_iso(), - "last_active": _now_iso(), - "message_count": 0, - "messages": [], - } - try: - await asyncio.to_thread(_write_json, path, skeleton) - logger.info("session_store: created session %s", session_id) - except Exception as exc: - logger.error("session_store: failed to create session %s: %s", session_id, exc) - - -async def save_message(session_id: str, question: str, answer: str) -> None: - """ - Appends a Question+Answer pair with timestamps to the session's JSON file. - Fire-and-forget safe — caller should wrap in asyncio.create_task(). - """ - path = _get_session_path(session_id) - logger.info("session_store: save_message called for session_id=%s, path=%s", session_id, path) - lock = await _get_lock(session_id) - async with lock: - try: - # Load current data - if path.exists(): - data = await asyncio.to_thread(_read_json, path) - else: - # Session file missing — recreate it gracefully - data = { - "session_id": session_id, - "user_name": "default name", - "created_at": _now_iso(), - "last_active": _now_iso(), - "message_count": 0, - "messages": [], - } - - timestamp = _now_iso() - data["messages"].append({ - "id": data["message_count"] + 1, - "timestamp": timestamp, - "question": question, - "answer": answer, - }) - data["message_count"] += 1 - data["last_active"] = timestamp - - await asyncio.to_thread(_write_json, path, data) - logger.info("session_store: successfully saved message to %s", path) - except Exception as exc: - logger.error( - "session_store: failed to save message for session %s: %s", - session_id, - exc, - ) - - -async def get_all_sessions() -> list: - """ - Returns a list of all session summaries (metadata only, no full messages). - Used by the Admin Panel sidebar. - """ - try: - files = await asyncio.to_thread(_list_json_files) - sessions = [] - for fpath in files: - try: - data = await asyncio.to_thread(_read_json, fpath) - sessions.append({ - "session_id": data.get("session_id", fpath.stem), - "user_name": data.get("user_name", ""), - "created_at": data.get("created_at", ""), - "last_active": data.get("last_active", ""), - "message_count": data.get("message_count", 0), - }) - except Exception as exc: - logger.warning("session_store: could not read %s: %s", fpath, exc) - # Sort newest first - sessions.sort(key=lambda s: s.get("last_active", ""), reverse=True) - return sessions - except Exception as exc: - logger.error("session_store: get_all_sessions failed: %s", exc) - return [] - - -async def get_session(session_id: str) -> dict: - """ - Returns the full session data (including all messages) for the Admin Panel. - Returns an empty dict if the session does not exist. - """ - path = _get_session_path(session_id) - try: - if not path.exists(): - return {} - return await asyncio.to_thread(_read_json, path) - except Exception as exc: - logger.error("session_store: get_session(%s) failed: %s", session_id, exc) - return {} - - -async def delete_session(session_id: str) -> bool: - """ - Deletes a session file. Returns True on success, False on failure. - Used by Admin Panel if needed in the future. - """ - path = _get_session_path(session_id) - lock = await _get_lock(session_id) - async with lock: - try: - if path.exists(): - await asyncio.to_thread(os.remove, path) - return True - except Exception as exc: - logger.error("session_store: delete_session(%s) failed: %s", session_id, exc) - return False - - -async def update_session_name(session_id: str, user_name: str) -> None: - """Updates the user_name field in the session JSON file.""" - path = _get_session_path(session_id) - lock = await _get_lock(session_id) - async with lock: - try: - if path.exists(): - data = await asyncio.to_thread(_read_json, path) - data["user_name"] = user_name - await asyncio.to_thread(_write_json, path, data) - logger.info("session_store: updated name for session %s to %s", session_id, user_name) - else: - # If session doesn't exist yet, we can't update it, but we could create it? - # Usually ensure_session is called first. - logger.warning("session_store: attempt to update name for non-existent session %s", session_id) - except Exception as exc: - logger.error("session_store: failed to update name for session %s: %s", session_id, exc) - - -async def rename_session(old_id: str, new_name: str) -> str: - """ - Renames a session file from old_id to a name based on new_name. - Returns the new session_id used. - """ - import re - # 1. Sanitize the name for use as a filename - clean_name = re.sub(r'[^\w\s-]', '', new_name).strip().replace(' ', '_') - if not clean_name: - clean_name = "User" - - new_id = clean_name - old_path = _get_session_path(old_id) - new_path = _get_session_path(new_id) - - # Ensure uniqueness: if John_Doe exists, try John_Doe_1, etc. - counter = 1 - while new_path.exists() and new_id != old_id: - new_id = f"{clean_name}_{counter}" - new_path = _get_session_path(new_id) - counter += 1 - - if new_id == old_id: - # Same filename — still update user_name in the JSON if it has changed. - # Without this, sessions whose file is already correctly named keep - # user_name = "default name" forever (the early return skips the update). - update_lock = await _get_lock(old_id) - async with update_lock: - try: - if old_path.exists(): - data = await asyncio.to_thread(_read_json, old_path) - if data.get("user_name") != new_name: - data["user_name"] = new_name - await asyncio.to_thread(_write_json, old_path, data) - logger.info( - "session_store: updated user_name in-place for %s → '%s'", - old_id, new_name - ) - except Exception as exc: - logger.error( - "session_store: failed to update user_name for %s: %s", old_id, exc - ) - return old_id - - # Lock both IDs to prevent race conditions during move - lock_old = await _get_lock(old_id) - lock_new = await _get_lock(new_id) - - async with lock_old: - async with lock_new: - try: - if old_path.exists(): - data = await asyncio.to_thread(_read_json, old_path) - data["session_id"] = new_id - data["user_name"] = new_name - - # Write to new path and delete old - await asyncio.to_thread(_write_json, new_path, data) - await asyncio.to_thread(os.remove, old_path) - - logger.info("session_store: renamed session %s -> %s", old_id, new_id) - return new_id - else: - logger.warning("session_store: cannot rename non-existent session %s", old_id) - return old_id - except Exception as exc: - logger.error("session_store: failed to rename session %s: %s", old_id, exc) - return old_id - - -# ── Sync helper functions (run via asyncio.to_thread) ──────────────────────── - -def _write_json(path: Path, data: dict) -> None: - """Atomically writes JSON to a file using a temp file + rename pattern.""" - tmp_path = path.with_suffix(".tmp") - with open(tmp_path, "w", encoding="utf-8") as f: - json.dump(data, f, ensure_ascii=False, indent=2) - os.replace(tmp_path, path) # Atomic on POSIX, best-effort on Windows - - -def _read_json(path: Path) -> dict: - with open(path, "r", encoding="utf-8") as f: - return json.load(f) - - -def _list_json_files() -> list: - return list(_get_sessions_dir().glob("*.json")) diff --git a/backup_2026-05-13_pre_intelligent_retrieval/app/services/vector_store.py b/backup_2026-05-13_pre_intelligent_retrieval/app/services/vector_store.py deleted file mode 100644 index e31029806d59d189af5f57cc6bb71d4759a74712..0000000000000000000000000000000000000000 --- a/backup_2026-05-13_pre_intelligent_retrieval/app/services/vector_store.py +++ /dev/null @@ -1,167 +0,0 @@ -import hashlib -import json -from pathlib import Path -from typing import Dict, List, Tuple - -import faiss -import numpy as np -import pickle -from rank_bm25 import BM25Okapi - -from app.services.chunker import chunk_documents -from app.services.document_loader import load_documents -from app.services.embeddings import EmbeddingService - - -class FaissVectorStore: - def __init__( - self, - embedding_service: EmbeddingService, - docs_dir: Path, - index_dir: Path, - chunk_size_tokens: int, - chunk_overlap_tokens: int, - ) -> None: - self.embedding_service = embedding_service - self.docs_dir = docs_dir - self.index_dir = index_dir - self.chunk_size_tokens = chunk_size_tokens - self.chunk_overlap_tokens = chunk_overlap_tokens - - self.faiss_index_path = index_dir / "faiss.index" - self.bm25_index_path = index_dir / "bm25.pkl" - self.metadata_path = index_dir / "metadata.json" - self.state_path = index_dir / "state.json" - - self.index = None - self.bm25 = None - self.metadata: List[Dict[str, str]] = [] - self.docs_loaded = False - self.last_retrieved: List[Dict[str, str]] = [] - - def _compute_docs_fingerprint(self) -> str: - hasher = hashlib.sha256() - # Include chunk settings in fingerprint so changing them triggers re-index - hasher.update(str(self.chunk_size_tokens).encode("utf-8")) - hasher.update(str(self.chunk_overlap_tokens).encode("utf-8")) - - if not self.docs_dir.exists(): - return "no_docs" - for path in sorted(self.docs_dir.rglob("*")): - if path.is_file() and path.suffix.lower() in {".txt", ".pdf"}: - stat = path.stat() - hasher.update(str(path).encode("utf-8")) - hasher.update(str(stat.st_mtime_ns).encode("utf-8")) - hasher.update(str(stat.st_size).encode("utf-8")) - return hasher.hexdigest() - - def _can_use_cached_index(self, fingerprint: str) -> bool: - if not (self.faiss_index_path.exists() and self.metadata_path.exists() and self.state_path.exists()): - return False - try: - state = json.loads(self.state_path.read_text(encoding="utf-8")) - return state.get("docs_fingerprint") == fingerprint - except Exception: - return False - - def build_or_load(self) -> None: - self.index_dir.mkdir(parents=True, exist_ok=True) - fingerprint = self._compute_docs_fingerprint() - - if self._can_use_cached_index(fingerprint): - self.index = faiss.read_index(str(self.faiss_index_path)) - with open(self.bm25_index_path, "rb") as f: - self.bm25 = pickle.load(f) - self.metadata = json.loads(self.metadata_path.read_text(encoding="utf-8")) - self.docs_loaded = len(self.metadata) > 0 - return - - docs = load_documents(self.docs_dir) - chunks = chunk_documents(docs, self.chunk_size_tokens, self.chunk_overlap_tokens) - if not chunks: - self.index = None - self.metadata = [] - self.docs_loaded = False - return - - vectors = self.embedding_service.encode([c["text"] for c in chunks]) - dim = vectors.shape[1] - index = faiss.IndexFlatIP(dim) - index.add(vectors) - - tokenized_corpus = [c["text"].lower().split() for c in chunks] - bm25 = BM25Okapi(tokenized_corpus) - - self.index = index - self.bm25 = bm25 - self.metadata = chunks - self.docs_loaded = True - - faiss.write_index(index, str(self.faiss_index_path)) - with open(self.bm25_index_path, "wb") as f: - pickle.dump(bm25, f) - self.metadata_path.write_text(json.dumps(chunks, ensure_ascii=False, indent=2), encoding="utf-8") - self.state_path.write_text( - json.dumps({"docs_fingerprint": fingerprint, "chunk_count": len(chunks)}, indent=2), - encoding="utf-8", - ) - - def search(self, query: str, top_k: int = 4) -> List[Dict[str, str]]: - if self.index is None or not self.metadata or self.bm25 is None: - self.last_retrieved = [] - return [] - - # Vector Search (FAISS) - query_vec = self.embedding_service.encode_query(query) - faiss_scores, faiss_indices = self.index.search(np.asarray(query_vec, dtype=np.float32), top_k * 2) - - # BM25 Search - tokenized_query = query.lower().split() - bm25_scores = self.bm25.get_scores(tokenized_query) - - # Get top indices for BM25 - bm25_top_indices = np.argsort(bm25_scores)[::-1][:top_k * 2] - - # Combine using Reciprocal Rank Fusion (RRF) - # RRF_score = 1 / (k + rank) - k = 60 - rrf_scores = {} - - # Add FAISS ranks - for rank, idx in enumerate(faiss_indices[0]): - if idx < 0 or idx >= len(self.metadata): - continue - rrf_scores[idx] = rrf_scores.get(idx, 0.0) + (1.0 / (k + rank + 1)) - - # Add BM25 ranks - for rank, idx in enumerate(bm25_top_indices): - if idx < 0 or idx >= len(self.metadata): - continue - rrf_scores[idx] = rrf_scores.get(idx, 0.0) + (1.0 / (k + rank + 1)) - - # Sort by combined RRF score - sorted_indices = sorted(rrf_scores.keys(), key=lambda x: rrf_scores[x], reverse=True)[:top_k] - - results: List[Dict[str, str]] = [] - for idx in sorted_indices: - chunk = self.metadata[idx] - # Fetch the original FAISS score for fallback relevance checking if needed, or just use RRF score - # Note: RRF scores are small (e.g. 0.03), so we must adjust the threshold in rag_pipeline - results.append( - { - "id": chunk["id"], - "source": chunk["source"], - "text": chunk["text"], - "score": float(rrf_scores[idx]), # RRF score - } - ) - self.last_retrieved = results - return results - - def health(self) -> Dict[str, object]: - return { - "docs_loaded": self.docs_loaded, - "index_ready": self.index is not None, - "chunk_count": len(self.metadata), - "index_path": str(self.faiss_index_path), - } \ No newline at end of file diff --git a/backup_2026-05-13_pre_intelligent_retrieval/app/ui_gradio.py b/backup_2026-05-13_pre_intelligent_retrieval/app/ui_gradio.py deleted file mode 100644 index 426bf5d006e060e8c5c94324fb2605019c24f485..0000000000000000000000000000000000000000 --- a/backup_2026-05-13_pre_intelligent_retrieval/app/ui_gradio.py +++ /dev/null @@ -1,275 +0,0 @@ -import asyncio -import logging -import os -import random -import string -import traceback -import uuid -from typing import Dict, List, Tuple - -from app.services import session_store as _ss - -import gradio as gr -import httpx - -from app.core.config import get_settings -from app.services.embeddings import EmbeddingService -from app.services.llm import LLMService -from app.services.rag_pipeline import RAGPipeline -from app.services.vector_store import FaissVectorStore -from app.services.reranker import RerankerService - -logger = logging.getLogger(__name__) -settings = get_settings() -API_URL = os.getenv("RAG_API_URL", "").strip() - -# Initialize services -embedding_service = EmbeddingService(settings.embedding_model) -vector_store = FaissVectorStore( - embedding_service=embedding_service, - docs_dir=settings.docs_dir, - index_dir=settings.index_dir, - chunk_size_tokens=settings.chunk_size_tokens, - chunk_overlap_tokens=settings.chunk_overlap_tokens, -) -llm_service = LLMService( - provider=settings.llm_provider, - groq_api_key=settings.groq_api_key, - groq_model=settings.groq_model, - groq_rewrite_model=settings.groq_rewrite_model, - hf_api_key=settings.hf_api_key, - hf_model=settings.hf_model, - timeout_s=settings.request_timeout_s, -) -reranker_service = RerankerService(settings.reranker_model) -pipeline = RAGPipeline( - vector_store=vector_store, - llm_service=llm_service, - reranker=reranker_service, - top_k=settings.top_k, - max_context_chunks=settings.max_context_chunks -) - -# Ensure index is ready -vector_store.build_or_load() - - -# ── Keyboard layout for realistic typos ────────────────────────────── -NEARBY_KEYS = { - 'a': 'sqwz', 'b': 'vghn', 'c': 'xdfv', 'd': 'sfecx', 'e': 'wrsdf', - 'f': 'dgrtcv', 'g': 'fhtybn', 'h': 'gjyunb', 'i': 'uojkl', 'j': 'hkunmi', - 'k': 'jlomi', 'l': 'kop', 'm': 'njk', 'n': 'bhjm', 'o': 'ipkl', - 'p': 'ol', 'q': 'wa', 'r': 'edft', 's': 'awedxz', 't': 'rfgy', - 'u': 'yihj', 'v': 'cfgb', 'w': 'qase', 'x': 'zsdc', 'y': 'tghu', - 'z': 'xas', -} - -TYPO_CHANCE = 0.07 # 7% chance per word - - -def _nearby_char(ch: str) -> str: - """Return a plausible neighbouring key for the given character.""" - lower = ch.lower() - if lower in NEARBY_KEYS: - replacement = random.choice(NEARBY_KEYS[lower]) - return replacement.upper() if ch.isupper() else replacement - return random.choice(string.ascii_lowercase) - - -def _to_history(messages: List[Dict[str, str]]) -> list: - """Returns the history as a list of dicts (already in this format for Gradio 5).""" - return messages - - -async def _chat_via_api(message: str, history: List[Dict[str, str]]) -> str: - payload = {"message": message, "history": history} - headers = {"Content-Type": "application/json"} - if settings.api_key: - headers["x-api-key"] = settings.api_key - - async with httpx.AsyncClient(timeout=30.0) as client: - resp = await client.post(API_URL, json=payload, headers=headers) - if resp.status_code == 429: - return "⚠️ I'm receiving too many requests right now. Please wait a moment before sending another message." - resp.raise_for_status() - return resp.json()["reply"] - - -async def _get_reply(message: str, chat_history: List[Dict[str, str]]) -> str: - """Fetch the full reply from the LLM (API or direct pipeline).""" - if API_URL: - return await _chat_via_api(message, chat_history) - result = await pipeline.chat(message=message, history=chat_history) - return result["reply"] - - -async def chat_fn(message: str, chat_history: List[Dict[str, str]], session_id: str = ""): - """Streaming generator: yields word-by-word with human-like timing & typos.""" - if not message or not message.strip(): - yield gr.update(interactive=True), chat_history - return - - original_history = chat_history.copy() - chat_history = chat_history + [ - {"role": "user", "content": message}, - {"role": "assistant", "content": ""} - ] - yield gr.update(value="", interactive=False), chat_history - - try: - reply = await _get_reply(message, original_history) - except httpx.HTTPStatusError as e: - logger.error(f"HTTP Error: {e.response.status_code} - {e.response.text}") - reply = "⚠️ Rate limit reached. Please slow down a bit!" if e.response.status_code == 429 \ - else "⚠️ I encountered an error. Please try again in a few seconds." - except Exception as e: - logger.error(f"Unexpected error: {str(e)}\n{traceback.format_exc()}") - reply = f"⚠️ Oops! Something went wrong." - - # ── Humanistic letter-by-letter typing — HTML-safe ──────────────────── - displayed = "" # what is shown in the chat bubble - buffer = "" # invisible accumulator for mid-HTML-tag characters - in_tag = False # True while we are inside a < … > sequence - - for char in reply: - if char == '<': - in_tag = True - buffer += char - continue - - if in_tag: - buffer += char - if char == '>': - # Tag is now complete — flush it to the display all at once - in_tag = False - displayed += buffer - buffer = "" - chat_history[-1] = {"role": "assistant", "content": displayed} - yield gr.update(value="", interactive=False), chat_history - # Small pause after a
to mimic line-break pacing - if displayed.endswith('
') or displayed.endswith('
'): - await asyncio.sleep(random.uniform(0.1, 0.2)) - continue - - # ── Normal character (not inside a tag) ─────────────────────────── - # Decide typo chance only for plain alphabetic words - do_typo = ( - char.isalpha() - and len(displayed) > 8 # skip the very beginning - and random.random() < TYPO_CHANCE - ) - - if do_typo: - wrong_char = _nearby_char(char) - displayed += wrong_char - chat_history[-1] = {"role": "assistant", "content": displayed} - yield gr.update(value="", interactive=False), chat_history - await asyncio.sleep(random.uniform(0.15, 0.3)) - - # Backspace the wrong character - displayed = displayed[:-1] - chat_history[-1] = {"role": "assistant", "content": displayed} - yield gr.update(value="", interactive=False), chat_history - await asyncio.sleep(random.uniform(0.1, 0.2)) - - # Type the correct character - displayed += char - chat_history[-1] = {"role": "assistant", "content": displayed} - yield gr.update(value="", interactive=False), chat_history - - # Pacing: punctuation pauses, space micro-pause, normal char speed - if char in '.!?': - await asyncio.sleep(random.uniform(0.35, 0.7)) - elif char in ',:;': - await asyncio.sleep(random.uniform(0.15, 0.3)) - elif char == ' ': - await asyncio.sleep(random.uniform(0.06, 0.13)) - else: - await asyncio.sleep(random.uniform(0.02, 0.06)) - - # ── Fire-and-forget: persist to session store (never blocks the UI) ─── - # We only save successful responses, not error messages - if session_id and not displayed.startswith("⚠️"): - asyncio.create_task(_ss.save_message(session_id, message, displayed)) - - # Re-enable the input box at the end - yield gr.update(interactive=True), chat_history - - -custom_css = """ -/* Aggressively force height: auto and min-height: 40px on the chatbot and flex containers */ -.gradio-container, .flex, .block, #chatbot-window { - height: auto !important; - min-height: 40px !important; -} - -/* Ensure the chatbot doesn't overflow the viewport if it gets too full */ -#chatbot-window { - max-height: 70vh !important; - border: none !important; -} - -/* Aggressively hide the footer, including HF injected footers if inside the container */ -footer, .footer, footer * { - display: none !important; - visibility: hidden !important; - height: 0 !important; - margin: 0 !important; - padding: 0 !important; -} -""" - -# JS: runs immediately on page load — generates/loads session_id from localStorage -# and restores previous chat history. Completely non-blocking. -_SESSION_JS = """ -async () => { - // ── Session ID: generate once, persist forever in localStorage ── - let sid = localStorage.getItem('martech_session_id'); - if (!sid) { - sid = 'sess-' + Date.now() + '-' + Math.random().toString(36).slice(2, 9); - localStorage.setItem('martech_session_id', sid); - } - // Write the session_id into the hidden Gradio textbox - const el = document.getElementById('session-id-box')?.querySelector('textarea'); - if (el) { el.value = sid; el.dispatchEvent(new Event('input', {bubbles:true})); } - return sid; -} -""" - -with gr.Blocks(title="Martechsol Assistant", theme=gr.themes.Soft(), css=custom_css) as demo: - gr.Markdown("# Martechsol Assistant") - gr.Markdown("Welcome! How can I help you today?") - - chatbot = gr.Chatbot( - show_label=False, - elem_id="chatbot-window", - render_markdown=True, - ) - with gr.Row(): - msg = gr.Textbox( - placeholder="Type your question here...", - show_label=False, - scale=9 - ) - send = gr.Button("Send", variant="primary", scale=1) - - clear = gr.Button("Clear Chat History") - - # Hidden textbox holds the session_id — invisible to the user - session_id = gr.Textbox( - value="", - visible=False, - elem_id="session-id-box", - label="session_id", - ) - - # On load: run JS to set session_id from localStorage (non-blocking) - demo.load(fn=None, inputs=None, outputs=session_id, js=_SESSION_JS) - - # Wire up the events — streaming generator for live typing effect (UNCHANGED) - send.click(chat_fn, inputs=[msg, chatbot, session_id], outputs=[msg, chatbot]) - msg.submit(chat_fn, inputs=[msg, chatbot, session_id], outputs=[msg, chatbot]) - clear.click(lambda: [], None, chatbot) - -if __name__ == "__main__": - demo.launch(server_name="0.0.0.0", server_port=7860, show_api=False) diff --git a/backup_2026-05-13_pre_intelligent_retrieval/docker-compose.yml b/backup_2026-05-13_pre_intelligent_retrieval/docker-compose.yml deleted file mode 100644 index 24d412fb9c78ed97ae3106fa07ef069bb5679c95..0000000000000000000000000000000000000000 --- a/backup_2026-05-13_pre_intelligent_retrieval/docker-compose.yml +++ /dev/null @@ -1,14 +0,0 @@ -version: "3.9" -services: - rag-api: - build: . - container_name: rag-api - ports: - - "8000:8000" - env_file: - - .env - volumes: - - ./docs:/app/docs - - ./data/index:/app/data/index - - ./data/sessions:/app/data/sessions - restart: unless-stopped diff --git a/backup_2026-05-13_pre_intelligent_retrieval/docs/MartechSol_Employee_HandBook (7) (1).pdf b/backup_2026-05-13_pre_intelligent_retrieval/docs/MartechSol_Employee_HandBook (7) (1).pdf deleted file mode 100644 index ae104106fb1a7f1745db014121416ba1ec57054c..0000000000000000000000000000000000000000 --- a/backup_2026-05-13_pre_intelligent_retrieval/docs/MartechSol_Employee_HandBook (7) (1).pdf +++ /dev/null @@ -1,3 +0,0 @@ -version https://git-lfs.github.com/spec/v1 -oid sha256:a48b14cc28d7892ae03a5dfe06a244832e2807ce178ca1f1acdbc58d6af4e8ef -size 3844502 diff --git a/backup_2026-05-13_pre_intelligent_retrieval/docs/MartechSol_Employee_Handbook_Extracted.txt b/backup_2026-05-13_pre_intelligent_retrieval/docs/MartechSol_Employee_Handbook_Extracted.txt deleted file mode 100644 index 5035befdbf23a1f6638cd933f3f855043175d423..0000000000000000000000000000000000000000 --- a/backup_2026-05-13_pre_intelligent_retrieval/docs/MartechSol_Employee_Handbook_Extracted.txt +++ /dev/null @@ -1,1091 +0,0 @@ -Employee Handbook -Last Modified: September 1st, 2024 - -Contents -Introduction ....................................................................................................... 4 -Workplace Orientation ....................................................................................... 4 -Employment Status .......................................................................................... 5 -Employment At Will .......................................................................................... 5 -Employee Responsibilities ........................................................................................ 5 -Workweek and Workday .................................................................................... 5 -Time Clocks and Time Cards ............................................................................... 5 -General Attendance Requirements ....................................................................... 6 -Work from Home .............................................................................................. 6 -Problem Resolution ........................................................................................... 7 -Personal Appearance ......................................................................................... 7 -Health Safety .................................................................................................. 7 -Employee Relationships ..................................................................................... 8 -Policy Against Harassment ................................................................................. 8 -Activity Permission ........................................................................................... 9 -Confidentiality ................................................................................................ 9 -Intangible Property ......................................................................................... 10 -Restraint of Trade, Non-Compete, and Cooling Off Period ....................................... 11 -Solicitation ................................................................................................... 11 -Cell Phone Usage ........................................................................................... 12 -Computer, E-Mail and Internet Policy ................................................................. 12 -Tobacco Use .................................................................................................. 13 -Grievance Procedure ....................................................................................... 13 -Copyright Violation ......................................................................................... 13 -Data Theft .................................................................................................... 13 -Separation from the Company .......................................................................... 14 -Exit Interviews .............................................................................................. 14 -Final Settlement ............................................................................................. 14 -Return of Property .......................................................................................... 15 - - -Acceptability ................................................................................................. 15 -Work Rules ................................................................................................... 15 -Level 1 Actions .............................................................................................. 16 -Level 2 Actions .............................................................................................. 16 -Level 3 Actions .............................................................................................. 17 -Disciplinary Procedures ..................................................................................... 18 -Employee Benefits ................................................................................................. 18 -Referral Program ............................................................................................ 18 -Game Room .................................................................................................. 18 -Holidays and Holiday Pay ................................................................................. 18 -Pay Period and Paychecks ................................................................................ 19 -Leaves ......................................................................................................... 19 -Sick Leaves ................................................................................................... 19 -Casual Leaves ............................................................................................... 20 -Annual Leaves ............................................................................................... 21 -Hajj Leave .................................................................................................... 22 -Maternity Leave ............................................................................................. 23 -Paternity Leave .............................................................................................. 23 -Bereavement Leave ........................................................................................ 24 -Unauthorized Leaves ....................................................................................... 24 -Unapproved Absence Without Pay ..................................................................... 25 -Leave Encashment ......................................................................................... 25 -Other Benefits ..................................................................................................... 25 -Loans & Advance Salary .................................................................................. 25 -Maternity (Pregnancy) Benefit .......................................................................... 26 -Company Maintained Vehicle ............................................................................ 27 -Healthcare Insurance – For Self & Spouse ............................................................ 27 -EOBI: .......................................................................................................... 30 -Provident Fund: ............................................................................................. 30 - - -Introduction -We are providing you this employee handbook to introduce you to our policies. The handbook will -describe some of the more important employee benefits and obligations. -Our business is constantly changing, and therefore MartechSol reserves the right to change all of our -policies, including those covered in this handbook, at any time. You will be informed of any changes -in policy through posting on portal or through other appropriate means. If you are uncertain about -any policy or procedure, please check with your Supervising Authority. -Workplace Orientation -Orientati -on Training and Initial Evaluation Period -All employees will receive orientation training at the beginning of employment with MartechSol. This -period of orientation will include familiarization with Logicose’s work policies, disciplinary policies, job -duties, and reporting responsibilities. -Upon completion of orientation, all employees will undergo evaluation during the first ninety (90) -calendar days of employment. If performance is unsatisfactory for any reason during this period of time, -the employee may be immediately dismissed from employment. At the end of this 90-day period, the -employee’s Supervising Authority will determine if the employee’s performance warrants continued -employment. -If the employee’s performance is not up to the Supervising Authority’s expectations in terms of quality of -work, output, behavior, or any other performance indicator, the employee will not be confirmed for -further employment at the company after this 90-day period. It is also up to the Supervising Authority to -end the probation period at any point before 90 days and either confirm or terminate the employee. -The Supervising Authority is required to make a decision regarding the confirmation of an employee after -90 days. If the Supervising Authority requires more time to evaluate an employee further, (s)he has the -right extend the probation period. -Confirmation -If performance exceeds expectations, an employee may receive a salary increment upon confirmation. -The date of confirmation depends on date of joining and performance. Revised salaries for employees -that are confirmed before the 15th of a given month will be effective from the 1st of that same month. On -the other hand, revised salaries will be effective from the 1st of the following month for employees that -are confirmed after the 15th of a given month. For example, if someone is confirmed on the 10th of April, -the revised salary will be effective from the 1st April, however if they are confirmed on the 16th of April, -the revised salary will be effective from the 1st of May. -Casual and sick leaves will also be assigned on a pro-rata basis based on this calculation. - - -Employment Status -A “regular full-time employee” is any employee who is employed to work on a year-round basis, and who -is regularly scheduled to work their full shift hours. -Employment At Will -All employees are employed on an "at-will" basis. This means that employees are not employed for any -specific term or period of time, and that the employer may terminate the employment relationship at -any time for any reason. Similarly, employees may resign and terminate employment with MartechSol -at any time and for any reason with prior notice of 30-90 days depending on their role within the -organization. -N -o employee has any authority to change the nature of this relationship through any statements, -representations, or conduct. The at-will relationship may be changed only through written and signed -authorization of the Supervising Authority. -Employee Responsibilities -Workweek and Workday -The nor -mal workweek shall be considered full shift hours, six days per week, from 9:00 a.m. to 5:30 p.m. -The first Saturday of every month is off and the remaining three Saturday’s in each month are six and a -half hours (half-day). One hour is allocated for lunch/dinner in all shifts. - Morning Shift: 1:30 pm - 2:30 pm - Evening Shift: 9:30 pm - 10:30 pm -Smoking Breaks: There are a maximum of 3 breaks allowed in a workday (other than lunch/dinner -break). These breaks are not to be longer than 5 minutes. -If any other break is required, the employee is to contact his or her Supervising Authority for approval. -Time Clocks and Time Cards -All empl -oyees shall use the portal to record their starting time and ending time. For wage payment -purposes, time worked will be figured to the nearest tenth of an hour. -Sharing login information (username and password) is strictly prohibited. Each person is to use their own -login information to time in at work. If any discrepancy is observed in the time in or time out process, -harsh disciplinary action will be taken against the person or persons involved. -In case an employee forgets to time in or is unable to time in, they are to submit a trouble ticket to the -Human Resources department. It is the employee’s responsibility to maintain their time sheet accurately. -Monthly salary will be calculated according to the time sheet. -Employees who want to change their work shift should contact their Supervising Authority for approval. -Necessary paperwork and legitimate reason are required for the shift change process. The decision to -change the shift of an employee is at the discretion of the Supervising Authority. - - -General Attendance Requirements -Regular and pro -mpt attendance of employees is essential to our business. Employees who cannot meet -our attendance requirements will be subject to counseling and disciplinary action. Excessive absenteeism -or tardiness will result in termination of employment. The Supervisor has the authority to mark a half -day, day off, or full day at his/her discretion. - Late Coming: Late coming is permitted till 20 minutes from your shift start time. If you arrive -after 20 minutes of your shift start time, then it will be considered as a late arrival. 4 late arrivals -in a month will be considered as a half day off. In addition, an employee is to inform his/her -Supervising Authority if they will be more than 30 minutes late to work. - Early Out: If you need to leave before your shift ends, verbal approval is mandatory from your -Supervising Authority. There will be no deductions for early out. - Half Day: Half day will be marked if you arrive 2 hours after your shift start time. If you complete -less than six and a half hours on any given day, then it will also be considered as a half day off. - Full Day: Employees who spend less than 2 hours at work on any workday will be marked absent -that day. - Employees who are late to work on any given workday are still responsible to complete all the -work assigned to them. If they are unable to, they should contact their Supervising Authority -immediately. If an employee arrives at work after 1 hour of their shift start time, by default, -the 8.5 hours shift has to be completed. If an employee is more than 1 hour late and wants - to -leave early (without completing the total (8.5 hr) shift time), he/she has to contact their -Supervising Authority for approval. - For -employees who are unable to complete at least 50% of their work during a workday, the -Supervising Authority reserves the right to decide the status of the workday (full day, off day, -half day) is at his or her discretion. -Work from Home -In case o -f emergency (bad weather, temporary disability, unrest in city, etc) when employees cannot -reach work, they will be requested to work from home. In case an employee has personal valid reason(s) -as to why they cannot reach work, they can work from home requesting approval from their Supervising -Authority. The approval of this request is at the Supervising Authority’s discretion. -If less than 30% of the work has been completed, the day will be counted as a full day off. If 30-70% of -the work has been completed, it will be counted as a half day. If more than 70% of the work has been -completed, it will be counted as a full day. -Work can be submitted no later than 3 hours after the end of the employee’s regular shift timing. If the -work is not submitted within the given time, it will not be marked as work provided for the same day. If -an employee faces problems submitting their work within the given duration and wants to submit their -work later, approval is required from their Supervising Authority. - - -Problem Resolution -If an emplo -yee faces difficulty or problems related to anything within the workplace, it is their -responsibility to submit a trouble ticket through the Portal. There is no need to contact the Supervising -Authority unless their ticket has not been addressed for more than 24 hours. -You are requested to use this trouble ticket system for your computer, admin and HR related -issues. How to use this system: -1. If you have any computer, admin, or HR related issue, go to the tickets link in the portal and -select ‘Submit a ticket’. -2. Select the concerned department for your problem e.g. if your mouse, keyboard, CPU or LCD is -not working properly, please select Networking. If there is any issue with your attendance or -payroll, select Human Resources. If there is any issue with fan, AC, kitchen item etc. please select -Admin department. -3. Once your trouble ticket is submitted, you may view the status of your ticket from ‘View open -tickets’ link. -As soon as your ticket is submitted, the Company will make every possible effort to resolve your issue on -an urgent basis. -There is also a messaging feature in this trouble ticket system. If there is any update related to your -ticket, you will receive a message within your open ticket. Similarly, if you want to send any message -related to your problem, you can also do that from ‘View open tickets’ section. -Once your issue is resolved, your ticket will be closed. You may view your closed tickets from ‘View your -closed tickets’ section. -This system will help resolve your issues quickly and will allow you to view status of your open tickets. -Personal Appearance -A company - is judged by its employees, as well as by its products and services. All employees are -expected to maintain good standards of personal neatness, including hair, clothes, and shoes. -Casual to business-style dress is appropriate. Employees should be neatly groomed and clothes should be -clean and in good condition. Leisure clothes such as cut-offs or halter tops are not acceptable attire for -the business office. -Health Safety -If an employee has had a contagious illness or serious health issue which may affect the health of other -employees (e.g. chicken pox, flu), a doctor’s written permission to return to work will be necessary. - - -Employee Relationships -MartechSol employs a number of people and, as a result, you will be working with people whose -personalities may differ from yours. It is your responsibility to get along with all of your co-workers. A -spirit of cooperation shown by all of us will help to accomplish our purpose of providing new -customers with the best possible service. -This does not mean that you need to become a personal friend of all your co-workers, but it does mean -that you should treat your co-workers with respect. Regardless of your personal opinions, your fellow -employee should be treated exactly as you would like to have that person treat you. -Senior employees should remember that they were once new employees and therefore should do -everything possible to aid new employees in acclimating themselves to their new jobs. -Policy -Against Harassment -MartechSol is committed to maintaining a work environment that is free of discrimination and -harassment. In keeping with this commitment, we will not tolerate harassment of our employees by -anyone, including any Supervising Authority, co-worker, vendor, client, or customer. -Harassment consists of unwelcome conduct, whether verbal, physical, or visual, that is based upon a -person’s protected status, such as sex, color, race, ancestry, religion, national origin, age, physical -handicap, medical condition, disability, marital status, veteran status, citizenship status, or other -protected group status. -MartechSol will not tolerate harassing conduct that affects tangible job benefits, that interferes -unreasonably with an individual’s work performance, or that creates an intimidating, hostile, or -offensive working environment. -Sexual harassment deserves special mention. Unwelcome sexual advances, requests for sexual favors, -and other physical, verbal, or visual conduct based on sex constitute sexual harassment when (1) -submission to the conduct is made a term or condition of employment, (2) submission to or rejection of -the conduct is used as the basis for an employment decision, or (3) the conduct has the purpose or effect -of unreasonably interfering with an individual’s work performance or creating an intimidating, hostile, or -offensive working environment. Sexual harassment may include explicit sexual proposition, sexual -innuendo, suggestive comments, sexually oriented “kidding” or “teasing,” “practical jokes,” jokes about -gender-specific traits, foul or obscene language or gestures, displays of foul or obscene printed or visual -material, and physical contact, such as patting, pinching, or brushing against another’s body. -All employees are responsible for helping to assure we avoid harassment. If you feel that you -have experienced or witnessed harassment, you should immediately notify your Supervising -Authority. MartechSol forbids retaliation against anyone who has reported harassment. -It is our policy to investigate all such complaints thoroughly and promptly. To the fullest extent -practicable, MartechSol will keep complaints and the terms of their resolution confidential. If an -investigation confirms that harassment has occurred, MartechSol will take appropriate corrective -action, including discipline up to and including immediate dismissal from employment. - - -Activity Permission -MartechSol is concerned with outside activities of employees only where such activities might involve a -conflict of interest with Logicose’s best interests. A conflict of interest exists if an employee has a -private financial interest or other relationship outside MartechSol that is detrimental to the best -interests of MartechSol. -You shall not without prior written consent of the company, engage directly or indirectly, in any trade, -business or occupation or any other income generation activity. -If an employee desires to attend an academic institution during his/her employment, (s)he must receive -permission from the company by submitting relevant documentation. It is mandatory to fill the Activity -Permission form for approval. -An employee should discuss any possible conflict of interest with the HR Department as full disclosure -and open knowledge are essential. For the employee’s own protection, written approval should be -obtained in all cases where a possible conflict of interest is involved. -Other possible conflict of interest situations may include any type of “moonlighting work” or a second -job, which interferes with the employee’s work for MartechSol, any type of external business -relationship with MartechSol, any of its competitors or any of its customers, and any type of public -activity which may affect adversely on MartechSol. -Confidentiality -MartechSol is a service provider that creates, completes, and delivers work for clients that do not want -our involvement disclosed in their projects. The ownership of the work we create for our clients is -transferred over to them and they hold the copyright to the work delivered. Disclosing such work may -result in Copyright Infringement, so it is in your, and the company’s best interest, to not share -involvement in such work. -Moreover, employees are prohibited from maintaining records of confidential information or intellectual -property on personal e-mail accounts, computers, hard drives, smart phones, cloud, or online. -Both during employment with MartechSol and after separation, employees are not permitted to -disclose to any third party or share any confidential information that is made available to you. -Confidential information includes, but is not limited to: - Web copy, eBooks, Press Releases, Reports, Infographics, Articles, Blogs, and/or any copy that was -created or revised for clients while employed at MartechSol is the property of either the clients or -MartechSol. Disclosing involvement in such projects to those outside the company, mentioning it -on your CV or resume, and/or making public your involvement in such projects is prohibited. - Search Engine Optimization strategy, campaigns, processes, proposals, accounts for posting, list -of clients, contact information of clients, content created or revised for client is confidential -information. Mentioning your involvement and/or disclosing such information to those outside -the company or on your CV or resume is considered breach of policy. - Client Information, leads, pricing strategy, proposals, account information, portal information, -samples, and promotional campaigns are the intellectual property of MartechSol. They may -neither be disclosed nor shared with those outside the organization. - - - Sensitive company information, such as employee contact details, software code, promotional -campaigns, client list, project(s) details, performance evaluation system, and in-house software. - Any and all such information is either Logicose’s or its Client’s intellectual property. -Immediately upon termination of employment, or at any other time upon the company’s request, all -employees will return to the Company: - All memoranda, notes and data, and computer software and hardware, records or other -data/documents compiled by the employee or made available to the employee during the -employee’s employment with the company concerning the business of the company, its -affiliates, or their customers. - All other Confidential Information that is disclosed to you during employment. - All physical and intellectual property of the company or its affiliates, including without limitation, -all the content, information, plans, files, records, documents, lists, equipment, supplies, -promotional materials, keys, vehicles, and similar items and all copies thereof or extracts there -from. -Disclosure of such information while employed or after separation from MartechSol is unethical, and -such actions will be considered intellectual property theft. -Furthermore, you cannot claim ownership of any project or developed content/material in any manner. -For example, giving a reference on your CV to a specific website for which you developed content while -working at the Company is prohibited. Any project or client details relating to your work carried out at -this company cannot be mentioned on your CV. -Intangible Property -During your course of employment with MartechSol, you will come across information that is privy to -those outside the organization. Such information is the intellectual (intangible) property of the company. -You may not at any time, during or after employment with MartechSol, have or claim any right, title, or -interest in any trade name, trademark, patent, copyright, work for hire, or other similar rights belonging -to or used by the company and shall not have or claim any right, title, or interest in any material or -matter of any sort prepared for or used in connection with the business or promotion of the company, -whatever your involvement with such matters may have been, and whether procured, produced, -prepared, or published in whole or in part by you. -MartechSol, or its clients, have and will retain the sole and exclusive rights in any and all such trade -names, trademarks, patents, copyrights, material, and matter unless transferred over to a third party. -Any and all documents, software, or copy that was created for MartechSol, its clients, or its affiliate -brand names, is the intellectual property of the company or its clients. You may not claim ownership, -maintain personal records of, or disclose your involvement in the creation or modification of such -information, software, or projects. - - -Restraint of Trade, Non-Compete, and Cooling Off Period -Any current or separated employee is prohibited from incorporating a new company or starting a new -business in the same industry before the time of at least 24 months. -Employee shall not, on his or her own behalf or behalf of others, hold any ownership interest in any -business engaged in the following industries: - Copywriting - Online Marketing - SEO - SMO - Ghostwriting - Industries that MartechSol, or any of its brand names, are currently or in the process of operating in -Employee agrees and covenants that the use of sensitive and confidential information that is made available, -in certain circumstances, may cause irreparable damage to MartechSol and its reputation. Therefore, -employee shall not until the expiration of two years after the termination of the employment, -through any corporations or associates in any business, form a business that is directly competitive with -Company for a period of two years. -Since the region or territory in which MartechSol operates is based online, the jurisdiction of the -restraint of trade is not limited to a specific city, state, or country. Forming an organization that is based -in any city MartechSol operates in, and/or creation of an online business is a breach of policy. -Failure to comply with activity permission, restraint of trade, confidentiality, intangible property, non- -disclosure, and non-compete will result in immediate termination. The Company also reserves the right -to take appropriate legal action upon breach of policy both during employment and after separation. -Solicitation -Both during employment with the company and for a period of two (2) years following the termination/ -resignation of employment with the company at any time and for any reason, employee’s will not, directly -or indirectly, on the employee’s own behalf or on behalf of any other person or entity, hire or solicit to -hire for employment or consulting other provision of services, any person who is actively employed or -engaged (or in the preceding six months was actively employed or engaged) by MartechSol. -This includes, but is not limited to, inducting or attempting to induce, or influencing or attempting to -influence, any person employed or engaged by the company to terminate his or her relationship with the -company. MartechSol reserves the right to take appropriate legal action against solicitation of employees. -Furthermore, both during the employee’s employment and after separation from the company at any -time and for any reason, the employee will not directly or indirectly, on the employee’s own behalf of -any other person or entity, solicit the business of any Customer or any other entity with which -MartechSol has worked with or discussed the possibility of any work, at the time of employee’s -termination, to provide services to such entity (a “Contractor”). Client information is confidential, and -MartechSol reserves the right to take legal action upon disclosure of privy information. - - -Cell Phone Usage -Emp -loyees shall not use cell phones for personal purposes while engaged in work activities. If the -employee needs to use the cell phone for important personal reasons, the employee shall excuse -themselves from the premises. Keep your cell phone on silent/vibrate. Use it responsibly, and make sure -that you do not disturb others while answering your calls. -Computer, E-Mail and Internet Policy -MartechSol maintains certain policy requirements for use of its “electronic communication systems.” -“Electronic communication systems” include computers, file servers, electronic information storage -systems, internal and external electronic mail systems, voice mail systems, local area networks and -any other system or technology used now or in the future by MartechSol to store or transmit data of -any kind electronically. -Our - policy requires the following: -1. Employees shall use the electronic communications systems for business purposes. -2. Logicose’s electronic communications systems and all information transmitted or received by, or -stored or contained in, these systems are the property of MartechSol. No employee shall have -any property rights in or expectation of privacy for any information, even if of a personal nature, -communicated or received through Logicose’s electronic communications systems. MartechSol -may review, delete and copy any such information without first obtaining the consent of the -employee who created, sent or received the information. No employee, including system -administrators and Supervising Authority, shall use any electronic communications systems for -purposes of satisfying idle curiosity about the affairs of others. There must be a business purpose -for obtaining access to the files or communications of others. -3. No employee shall use the electronic communications systems in a manner that constitutes a -violation of provincial or federal law or that is offensive to others. -4. Due to the threat to system stability posed by computer viruses and system incompatibility, no -employee shall install or download software from the Internet or elsewhere on his or her -computer without the supervision of the office administrator. -If you have any questions about the policy or its administration, please contact your Supervising -Authority. Violations of the policy may result in disciplinary action up to and including termination. -USBs cannot be used on any office computer. If necessary, please contact the System Administrator. -The completion of assigned work is the employee’s responsibility. If an employee has any issues -regarding their PC or internet, they are to raise a trouble ticket immediately. -The use of the following is prohibited: - Social media websites (e.g. Facebook) - Chatting websites (e.g. Meebo) - Chatting applications (e.g. MSN Messenger, Skype) - Online gaming websites (e.g. miniclips.com) - E-mail sites (e.g. Hotmail) - - - Political Websites - Pornography -Tobacco Use -MartechSol maintains a smoke-free workplace for all of its employees, which is why smoking is -prohibited in the workplace. Employees who must smoke are encouraged to do so away from Company -property and during work and/or meal breaks only. -Grievance -Procedure -MartechSol has a grievance procedure to make sure that problems and complaints an employee has -concerning his or her treatment or working conditions is called to our attention. This does not -include computer-related issues or similar technical issues. For such problems, the trouble ticket -system is provided. Employees with a problem should take the following steps: -1. Explai -n the problem to your immediate Supervising Authority. Allow 48 hours for an answer. -2. If you are still not satisfied, put your grievance in writing and submit it to MartechSol President via -the Feedback Form on the Portal. If possible, a decision will be given within three working days. -Copyright Violation -Any work performed within the premises of MartechSol for its clients, employees, or the Company itself -is under the sole ownership of MartechSol or those that the company transfers the rights to. Employees -cannot claim ownership of this work or use it outside company premises for other purposes, this -includes creating personal records on computers or e-mail accounts that aren’t the property of -MartechSol. Employees cannot use this work as reference to anyone else either, unless they get written -consent from MartechSol. Listed below are some examples of copyright violation: -Showing, sharing, using, creating a personal record of, or claiming ownership or involvement in the -creation of: - Co -py written - Software developed - Client contact information - Payments received at MartechSol - In -tellectual property -Data Theft -Any data relating to MartechSol, its clients, or employees is under the sole ownership of MartechSol. -Employees cannot use or transmit this data, as it is not under their ownership. Employees cannot use -this data as reference either. -Archiving such work on external hard drives, online, cloud, and/or on computers that are not property -of MartechSol is a breach of policy. - - -Separation from the Company - Prio -r to Confirmation: -o Minimum one day notice period, or as communicated in the offer letter, is required -during probationary period on behalf of the employee. -o If employee performance/behavior is not up to Company expectations, then the -Supervising Authority has the right to end the probation period and terminate the -employee. - After Confirmation: -o The employment relationship may be terminated either by you or by the company with a -prior written notice of one month or salary in lieu thereof, unless mentioned below. It is, -however, understood that no notice is required in case your services are terminated on -grounds of misconduct, willful neglect of duty, breach of trust or any other dereliction of -duty or misdemeanor prejudicial to the interest of the company. It is further understood -that any financial indebtedness of yours, standing to the detriment of the company’s -prestige will amount to a very serious misdemeanor towards the company -o A minimum notice period of at least 30 days is to served unless listed below otherwise: -Managerial Roles -Vice Team Lead 30 days -Team Lead 60 days -Group Team Lead 90 days -Exit Interviews -All employees separating from MartechSol must complete an exit interview with their immediate -supervisor. An exit interview gives the Human Resource Department the opportunity to obtain feedback -from employees as they transition out of their positions. -Final Settlement -All employees must undergo a final settlement before they separate from the organization. All -adjustments will be made according to below mentioned procedures: - Bef -ore Salary is Processed: -If any employee resigns before the salary is processed of any given month, their adjustments -(loans, advances, pro-rata leaves, etc.) will be made in the same month and the salary will be -deposited along with the rest of the employees. The remaining amount of the following month’s -salary will be processed as per the payroll process. No adjustments will be made in the following -salary except for early exit. For example, someone resigns on the 20th of April and his last working -day is the 20th of May, the adjustments will be made in the month of April’s salary. The salary for -the remaining days of the notice period to be served in May will be processed with May’s payroll -with Early Exit deduction of 11 days (21st May-31st May). - After/At the time Salary is Processed: -If any employee resigns once the salary is processed of any given month, their adjustments (loans, -advances, pro-rata leaves, etc.) will be made in the following month and the salary will be -deposited along with the rest of the employees. For example, someone resigns on 25th of April and - - -his last working day is on the 25th of May, the adjustments will be made in the month of May’s -salary. -Return of Property -Employees who separate from MartechSol service must return all issued property such as identification -cards, health insurance card, company maintained vehicle, and intellectual property of the organization. -Acceptability -Any changes in Policies, Rules, and Code of Conduct will first be shared through the MartechSol Portal. -A reasonable period of time will be allotted to discuss suggestions and amendments in changes before -its implementation. -Timing In to work after date of implementation implies that you understand and will abide by -organization policy. -Work -Rules -Any work performed within the premises of MartechSol is under the Company’s ownership and/or its -clients. Employees cannot claim ownership of any work done in or for MartechSol, neither can they use -it in the future. -All work performed at MartechSol has to be done in a legal and ethical manner so there is never -any copyright violation. -Employees shall perform their assignments within the specifics of the position description. Employees -who consistently fail to conform to the specifics of their position description or exhibit inappropriate -behavior or poor performance shall be required to meet with their supervisor. -This meeting will attempt to identify the problems, find ways to improve the situation and suggest -adequate solutions, concluding with a recommended course of action and an appropriate time frame -in which the employee will be expected to improve to the satisfaction of MartechSol. Details of the -meeting will be documented, signed by all parties as a correct representation of points discussed and -placed in the employee’s personal file. -Disciplinary action and corrective measures are taken at the discretion of the Management at -MartechSol. The possible steps for disciplinary action are: -1. Co -unseling/Verbal Warning -2. One or more formal written warnings through email -3. Issuance of Notice -4. Suspension -5. Dismissal -The choice of options depends on the seriousness of the behavior. Exceptions or deviations from the -progressive discipline sequence may occur whenever the Supervising Authority, in conjunction with the -President/Director, deems that circumstances warrant. -For level 1 offense, discussions between the employee and his or her supervisor will occur to allow the -employee to correct the situation. When a warning notice is issued, it becomes a part of an employee's - - -record and is considered when evaluating an employee for promotion, transfer, training or additional -discipline. Various offenses have been divided into different levels according to severity. -Three warning notices within twelve (12) months’ time, regardless of the type of level offense, will result -in discharge. -Level 1 Actions -These actions are taken for behaviors such as: - Unautho -rized or excessive absence, tardiness or early timing out - Unauthorized time away from work station - Slowing or interfering with the work of other employees - Failure to notify supervisor of incompletion of assigned work before leaving - Obscene, abusive, harassing, or disruptive language or behavior - Failure to perform assigned job responsibilities - Failure to follow prescribed work procedures - Failure to notify supervisor of absences - Neglect of organization property - Excessive personal use of cell telephone and/or email - Work shift not completed -Procedure For Dealing With Level 1 Behavior: These procedures are at the discretion of the Supervising -Authority. - Counseling/Verbal warning - Formal written warning in the form of email - Issuance of notice - Suspension -Level 2 Actions -These are more serious and must be dealt with firmly and immediately. Typical behaviors in this level -include: - Reoccurrin -g tardiness without reasonable explanation - Absences without approved leave - Refusal to comply with instructions of a supervisor - Conduct endangering the safety of the employee, co-workers or members - Working when ability is impaired by the use of alcohol, and/or illegal drugs - Unscheduled leaving from the workplace without informing supervisor - Habitual sleeping during work hours - Unauthorized use of organization materials and supplies - Fighting or threatening violence in the workplace - Unauthorized possession of weapons on organization property - - - Knowingly timing in another employee - Sharing portal login info - Inappropriate usage of internet, or any usage of internet unrelated to work during work hours - Harassing or bullying coworkers -Procedure for dealing with Level 2 behavior: These procedures are at the discretion of the Supervising -Authority. - Issuance of Notice - Suspension or probation - Termination -Level 3 Actions -These are b -ehaviors that are serious enough to justify either a suspension or, in extreme situations, -termination of employment without following the preceding disciplinary steps. Behaviors for which -immediate termination can be justified include, but are not limited to, the following: - Sexual harassment - Involvement in any income-generating activity without the consent of supervising authority - Insubordination, or the refusal to comply with the specific instructions of a supervisor in the -context of an assigned job duty - Falsification of personnel records, time records, or any other organization documents and records - Fighting during work time or on work premises - Use of, or possession of, alcohol or illegal drugs during work time or on work property - Damaging, defacing, or misusing organization property or the property of coworkers - Theft, misappropriation, embezzlement, unauthorized possession or removal of organization -property or the property of employees or customers - Immoral or indecent conduct which occurs on organization property - Possession of explosives, firearms, or other dangerous weapons on work premises - Failure to report an absence for a three-day period without a satisfactory explanation - Unauthorized release of work-related data or information - Continued unsatisfactory job performance - Violation of the organization's conflict of interest/ethical standards - Data theft: Stealing any digital or other type of data (e.g. email, company records, client records) -which belongs to MartechSol. - Copyright violation: Showing ownership of specific work performed at MartechSol. Discussion -of clients, websites, revenue generated, payments, or any work-related details outside the -Company’s premises. - Ot -her behaviors that, in the opinion of the Supervising Authority, seriously threaten the well- -being of co-workers. -In case of any ambiguity, please contact your supervising authority. - - -Procedure for dealing with Level 3 behavior: These procedures are at the discretion of the Supervising -Authority. - Suspension - Termination -Disciplinary Procedures -MartechSol may use progressive discipline as a means of enforcing minor violations of the work rules. -Progressive discipline may include steps such as counseling, written warning, and suspension. -Depending on the nature of the misconduct and the position involved, steps in the process may be -omitted or employment may be immediately terminated. The decision whether to apply progressive -discipline in any particular situation will rest within the sole discretion of management. -Examples of misconduct which will usually result in immediate discharge include theft, fighting or -physical assault, insubordination, dishonesty, falsification of time cards, production records, or other -Company documents, possession of alcohol or illegal drugs in the workplace, and disloyalty or breach of -confidentiality. This list is not intended to limit our right to discharge in other situations when we believe -termination of employment is appropriate. -Employee Benefits -Referral Program -Any employ -ee can benefit from the referral program by referring qualified friends or acquaintances. -The employee who has referred an individual is eligible for a PKR. 5,000 bonus once the new -candidate joins the organization. -Game Room -In order to promote a relaxed environment and provide employees with more forms of -entertainment, MartechSol has set up a game room in Room 301. -This game ro -om is accessible during lunchtime, so the morning shift has access to it from 1:30-2:30 pm -and the evening/night shift has access from 9:00-10:00pm. The game room can also be used after work. -Females are to use the game room on Monday, Wednesday, and Friday during lunchtime. Males should -use the game room on Tuesday, Thursday, and Saturday during lunch hour. -Holidays and Holiday Pay -MartechSol observes all Federal holidays as paid holidays. Due to our nature of work, employees -may be asked to work on specific holidays, according to their availability. They will be awarded -compensatory leave allowance or compensatory leave (employee’s choice), which they can utilize at -their convenience. -To qualify for holiday pay, an employee must have worked his or her regularly scheduled shift on the -day preceding the holiday and must work his or her regularly scheduled shift on the day following the -holiday. - - -Pay Period and Paychecks -For pay c -alculation purposes, the workweek (or “pay period”) will commence 12:01 a.m. on Sunday -morning and end at 12:00 p.m. on Saturday evening. Employees will be paid monthly on (payday), for the -preceding month. If payday falls on a holiday, then paychecks will be provided the day before the -holiday. -Employees are expected to immediately report any mistakes in their paychecks. Underpayments will be -corrected as soon as possible. If it is not practical to issue an employee a new paycheck to correct any -overpayment, the overpayment will be deducted from the employee's next paycheck. -Salary deduction for days off will be on “per-day” basis. {(Gross Salary x 12) / 365}. -Payroll for June and December -The salary for June and December will be processed and transferred during the first week of July and -January respectively. This is to ensure that the transition from the current to the revised salary is smooth -and all adjustments for the preceding month are processed correctly. -Leaves -Prior to Confirmation: - No leaves are allowed prior to your confirmation. - In case you skip a day, you will be marked absent, and your salary will be deducted for the day. - Prior approval from your supervising authority is required for situations where you would be -unable to come into work. - If you are absent from work as a result of sickness or any other unforeseeable personal matter, -you are required to notify your supervisory authority or the HR department by telephone or SMS -within two hours of your scheduled shift starting time. -After Confirmation: -Sick Leaves -Definition: -Sick Leave is defined as the period of time an employee is absent from work with full pay as a -result of an illness or injury. -Policy: - All full time confirmed employees are entitled for a maximum of 8 days of sick leave per annum. - Sick leaves cannot be carried forward to the next year and neither can they be encashed. - Each year on the 1st of January, your leave account will be credited with the sick leave -entitlement. - If anyone is confirmed in MartechSol during the year, then his/her sick leave entitlement will be -on a pro-rata basis. - In case of separation from the company during the year, sick leave availed will be adjusted on - - -pro- rata basis from your final salary payment. - If an employee takes sick leave from work starting any day of the week, any weekend or holiday -that falls in between will be counted as a full day sick leave. - Sick leave availed in excess of entitlement will be treated as casual or annual leave (if available -and subject to approval from reporting authority); incase of insufficient casual/annual leave -balance, excess sick leave will be treated as unpaid leave. - During notice period, no sick leave will be entertained even if the leave shows credit balance. -Process and Documentation: - Planned appointments, surgery, or any medical instance which can be deemed as anticipated is -subject to prior approval from your supervising authority. - If you are absent from work as a result of sickness or injury, you are required to notify your -supervisory authority or HR department by telephone or SMS within two hours of your -scheduled shift starting time (or before if possible). Failure to comply with the process may result -in absent without leave. - You must apply for sick leave immediately after returning from your sick leave by submitting -your leave application to the HR department after approval from your supervising authority. - The submission of a medical certificate or relevant documentation when applying for a sick leave -is mandatory, the leave will be considered absent without pay if proper documentation is not -submitted. - Sick leave can only be availed when sick and cannot be claimed for other purposes. Falsifying -information or using sick leave for a purpose other than illness or injury will make you liable for -disciplinary action. -Casual Leaves -Definition: -Casual leaves may be availed by employees in cases of urgent or unforeseen contingencies. -These cases include, but are not limited to, situations which may arise on account of unforeseen -circumstances e.g. birth/illness/death in the immediate family or of a close relative. -Any personal reason that prevents you from coming into work can be used for your casual leave. -Policy: - All full time confirmed employees are entitled for a maximum of 10 days casual leave per annum. - Casual leaves cannot be carried forward to the next year. - Casual leaves that have not been exhausted over the calendar year will be subject to encashment -with the paycheck of the following month. - Each year on the 1st of January, your leave account will be credited with the casual leave -entitlement if eligible. - If anyone is confirmed in MartechSol during the year, then his/her casual leave entitlement will -be on a pro-rata basis. - In case of separation from the company during the year, casual leave availed will be adjusted on - - -pro-rata basis from your final salary payment. - If an employee takes casual leave from work starting any day of the week, any weekend or -holiday that falls in between will be counted as a full day casual leave. - Casual leave cannot be taken for 3 consecutive days at any given time. If leave availed is for or in -excess of 3 days, the entire period will be treated as unpaid leave. However, if circumstances -warrant, the department head may approve the first 3 days to be treated as casual leave, and -only the additional days to be treated as annual leaves (if available). - Employees can take up to 2 consecutive casual leaves. If the number of casual leaves is 3 or -more, the whole duration will be treated as unpaid automatically unless approved by the line -manager. - In case of any urgent requirement, your reporting authority may ask you to report to work during -your casual leave. - Combining casual leave with a weekly holiday or a public holiday shall be inadmissible and that -additional leave shall be adjusted as leave without pay. For example, if a public holiday falls on -Friday and any employee takes leave on Saturday, all three days will be counted as casual leave. - During notice period, no casual leave will be allowed even if the leave shows credit balance. -Documentation and Process: - If you are absent from work as a result of any urgent personal responsibilities or any -unforeseeable circumstances, you are required to notify your supervisory authority or HR -department by telephone or SMS within two hours of your scheduled shift starting time (or -before if possible). Failure to comply with the process may result in absent without leave. - You must apply for casual leave immediately after returning from your casual leave by submitting -your leave application to the HR department after approval from your supervising authority. - In cases of planned or known circumstances, an employee must apply for casual leave to the -immediate supervisor for approval in advance. -Annual Leaves -Definition: -The purpose of annual leave is to provide employees with a reasonable period of rest and -a significant break from the workplace in order to maintain a healthy work life balance. -Policy: - All Full-time confirmed employee(s) shall be entitled to 14 days paid annual leave after -completion of one year of service. - Annuals leaves are not subject to encashment, except for TL roles or greater and for the Sales & -Support Department. - Up to 7 annual leaves can be carried forward to the next year, if approved by supervising -authority, for the following reasons:  - Marriage - Obligation outside the country (Hajj, Residence Visa Process, etc.) - Or for any planned reason that warrants extended leave.  - - - Carried forward annual leaves can’t be carried forward for more than a year.  - If any employee completes one year of service at MartechSol during the calendar year, then his/ -her annual leave entitlement will be on a pro-rata basis for that year. - In case of separation from the company during the year, annual leave availed will be adjusted on -pro-rata basis from your final salary payment. - Annual leave may be availed at one time during the year or may be divided and availed at various -times during the year, subject to approval. - Annual leaves will be granted in keeping with the operational exigencies of the organization, and -will be scheduled by the supervisor to meet the requirements of individual employees as far as -possible. - Management reserves the right to refuse a leave request, allow a partial request, revoke -approval if already granted, and/or to recall an employee before expiry of the leave period. - If you are hospitalized during your annual leave and produce a valid medical certificate stating -the fact, then these days will be adjusted as sick leave (if available). - If an employee takes annual leave from work starting any day of the week, any weekend or -holiday that falls in between will be counted as an annual leave. - During notice period, no annual leave will be allowed even if the leave shows credit balance. -Documentation and Process: - A completed leave application form is to be submitted for approval in advance to the HR -department prior to the anticipated start of the leave accordingly: -Number of leaves Approval time required -1-3 days Before 2 days -4-7 days Before 1 week -More than 7 days Before 1 month - An employee who wishes to obtain annual leaves shall apply in writing to HR in advance before -the commencement of leaves. - If the leave request is refused or postponed on account of company exigencies, the fact of such -refusal or postponement and reasons thereto shall be conveyed to the employee concerned. - If the requested leave is granted the employee will proceed on his leave on the approved date. - If an employee remains absent beyond the approved period of leave, he shall be liable for -disciplinary action unless the employee has justifiable reasons for absenteeism. - Annual leaves may only be availed after prior approval has been granted, however in case of an -emergency, annual leaves may be entertained subject to your supervising authority’s discretion. -Hajj Leave -Definition: -Employees who wish to make their pilgrimage to Mecca during Dhu'l Hijja may do so during -their tenure at MartechSol. These leaves are separate from the employee's annual leaves or any -other leave which he/she is entitled to. - - -Policy: - All full time permanent employees with 4 years of service are entitled for paid Hajj Leave for 10 -days. - Any leaves beyond 10 days needs to come from your annual or casual leaves. - Hajj leave will be granted only once during the whole employment period. - -Documentation and Process: - Your leave application must be submitted three months in advance for the necessary approval. - Copy of your Hajj Passport/Visa needs to be submitted to HR. -Maternity Leave -Definition: -A period of approved absence for female employees that wish to continue employment -at MartechSol after delivery, granted for the purpose of giving birth and taking care of -newborn children. -Policy: - All full-time female employees shall be entitled to 90 days paid maternity leave, taken no later -than two weeks before the expected delivery date and is only applicable after completion of one -year of service. - This entitlement shall be available only twice during the entire period of service. - Premature birth shall be considered as maternity leave. In case of still birth, female employee -shall be entitled to 30 days paid maternity leave. -Documentation and Process: - Maternity Leave form needs to be submitted and approved five months before planned -maternity leave. - -Paternity Leave -Definition: -A period of approved absence for male employees granted after or shortly before the birth of his -child. -Policy: - All full-time male employees are entitled to 3 days paid paternity leave. - This entitlement shall be available only twice during the entire period of service. - - - - Male employees may combine other leaves with paternity leave subject to approval from -supervising authority.  - Premature birth shall be considered as paternity leave. In case of still birth, male employee shall -be entitled to up to 3 days paid paternity leave. -Documentation and Process: - Application for paternity leave should be submitted to HR department at least two months in -advance supported by the gynecologist’s certification. -Bereavement Leave -Definition: -Bereavement leaves are paid leaves for employees that cover absences related to the death of -immediate family members. - Policy: - All full-time employee(s) shall be entitled to 3 days paid bereavement leave. - Bereavement leave will normally be granted unless there are unusual business needs or staffing -requirements. An employee may, with his or her supervisor’s approval, use any available leaves -for additional time off as necessary. - Documentation and Process: - You must apply for bereavement leave immediately after returning from your leave by -submitting your leave application to the HR department after approval from your supervising -authority. -Unauthorized Leaves - Any - absence without prior approval from your reporting authority will be considered as an -unauthorized leave. - All unauthorized and/or unreported absences shall be considered Absences without Leave, -(AWOL). Each unauthorized leave will result in a disciplinary notice and will be added in your file. -3 unauthorized leaves in a quarter will result in a severe disciplinary action. - In case you were not able to inform your reporting authority due to some unavoidable reasons, -you may discuss it with your reporting authority later. Based on discretion, he/she may authorize -your leave. - For employees serving their probationary period, failure to communicate with the respective -authority for more than 3 consecutive days of leave will result in termination. - For permanent employees, failure to communicate with the respective authority for more than 7 -consecutive days of leave will result in termination. - - -Unapproved Absence Without Pay -Definition: -Any leave, in excess of the 10 Casual leaves that you are entitled to, is an Absence without Pay. -These leaves, when taken without prior approval from your supervising authority are considered -Unapproved Absence without Pay. -Policy: - - Unapproved Absence without Pay will result in salary deduction(s) per day absent. - Taking more than two consecutive Absences without Pay prior to the weekend will result in salary -deduction for the weekend. For example, if an employee is absent on Thursday, Friday, and -Saturday then Sunday will also be marked absent. In other words, employees must report to work -on the last working day of the week to avoid further deductions.  -Leave Encashment -Each employee is entitled to a number of Sick, Casual, Annual, and other leaves. However, only Casual -leaves are subject to encashment for specific departments. -Listed below is breakdown of Department eligibility of leave encashment: -Department Casual Annual -Copywriting Yes No -SEO Yes No -Sales Yes Yes -Software Yes No -Networking Yes No -Human Resources Yes No -Admin Yes No -Other Benefits -Loans & Advance Salary -Definition: -The purpose of loans and advance salary is to meet an emergency need that can’t be -accommodated through other financial arrangements. Only genuine emergency needs such as -extraordinary medical costs not covered by insurance, or any other extreme financial or medical -emergency will be considered. The decision to approve or reject a loan/advance salary request will -be taken by a 3-person committee which will comprise of line manager/head of department, a -representative from HR, and a representative from finance. -Policy: - Decisions regarding loan requests will be relayed within 3 to 7 working days. For extremely urgent -matters, the committee will take a decision within 27 hours of request sent. - The decision of the committee will be final and non-negotiable. - A loans/advance salary can only be taken once in 6 months. - - - If an employee has an ongoing loan/advance salary, he/she will not be allowed to avail another -loan/advance salary. - Repayment period is up to 6 months. - Loan amount can’t exceed 3 months’ salary. - Advance salary amount shall not exceed from your monthly salary. -Documentation & Process: - If possible, the loan/advance salary form should be submitted at least a week (7 days) before it is -required. - The form can be acquired from HR and must be completed including all the signatures required. - The necessary documentation & supporting documents should be provided along with the filled -form. -Maternity (Pregnancy) Benefit -Definition: -This benefit provides financial assistance to female and male employees to cover the cost of childbirth and -other delivery related expenses. -Policy: - Male and female employees that have completed at least one year of service can avail maternity -benefit of PKR 50,000. - Maternity benefit can be utilized twice during tenure at MartechSol. - This benefit also covers stillbirth. -Documentation & Process: - Maternity Benefit Form and relevant documentation must be provided at least 60 days before -expected delivery. - Employees will be awarded this benefit a week before date of expected delivery. - In case of premature delivery, employees can either contact Supervising Authority or Admin -Department to arrange the benefit urgently or they may pay out of pocket at the time and avail the -benefit when they rejoin the office. - - -Company Maintained Vehicle -Definition: -MartechSol provides company maintained cars to employees at certain designation levels. The -policy varies across different departments. The company maintained car will be owned and -maintained by the company for the use of the employee. -Policy: - Employees should drive their allotted vehicle in a safe and responsible vehicle. They must abide by -driving laws. - Scheduled maintenance will be carried out by the company after every 5,000 KMs or 3 months -(whichever comes first). - Employees are requested to keep a copy of the following documents in the company vehicle: -o Authority Letter -o Motor Vehicle Tax paper -o Insurance paper -o Copy of the first page of the vehicle’s registration book -Documentation & Process: - Prior to vehicle allocation, employee must provide a copy of valid driving license of self (or driver) -for company record. - All maintenance and wear & tear requests need to be made through submitting a ticket on portal or -by contacting Admin Department. - In case of an accident: -o All accidents must be reported to the Admin Department. -o If the company vehicle is stolen, report the theft immediately to the local police and to the -Admin Department. Obtain a copy of the police report filed. -Healthcare Insurance – For Self & Spouse -Health care benefits are provided to all full-time employees and their spouse. Complete details of this -program are provided below: - “ANNEXURE A” -PROCEDURE FOR UTILIZATION OF HOSPITALIZATION EXPENSE BENEFIT -Emergency Hospitalization -In case of emergency, approach the nearest hospital. In case it is a PANEL HOSPITAL -1. Identify yourself as an NJI insured. -2. Produce Health Card / Letter of Authority -3. Get the treatment -4. Get the treatment on credit, up to the limits available. -5. Pay only the amount that exceeds the entitlement, if any, before discharge. - - -In case it is a NON-PANEL HOSPITAL -1. Inform NJI within 24 hours of the hospitalization. -2. Pay cash for the treatment. -3. Submit all original bills /supporting documents* with our claim form for reimbursement, -within 30 days of discharge from the hospital. -4. Settlement of claim will be done in line of the policy terms. - “ANNEXURE B” -PROCEDURE FOR UTILIZATION OF HOSPITALIZATION EXPENSE BENEFIT -Emergency Hospitalization -If hospitalization is advised by a licensed physician - In case it is a PANEL HOSPITAL -1. Approach any of our panel hospital with our Letter of Authority / Health Card -2. Identify yourself as an NJI insured -3. Get the treatment on credit, up to the limits available. -4. Pay only the amount that exceeds the entitlement, if any, before discharge. -In case it is a NON-PANEL HOSPITAL -1. Send us the cost estimate from the concerned doctor of the treatment with details, for prior -approval. -2. Get the treatment and pay cash for the treatment. -3. Submit all original bills /supporting documents* including our approval letter with our claim -form for reimbursement, within 30 days of discharge from the hospital. -4. Settlement of claim will be done in line of prior approval given and other policy terms. -*Supporting Documents: - Duly completed original NJI Claim Form - Original itemized bill/invoice (breakup of charged) on hospital -bill-book - Discharge card / clinical summary - Copy of NJI card or letter of admission - Diagnostic reports - Doctors prescriptions - Original pharmacy vouchers - Original payment receipts - - -JUBILEE GENERAL INSURANCE COMPANY LTD -GROUP HEALTHCARE INSURANCE PROPOSAL FOR -MartechSol -Hospitalization & Related Benefits -Plan A Plan B Plan C Plan D -H&R Limits (Per Person / Per Year) Rs.500,000 Rs.400,000 Rs.300,000 Rs.150,000 -Room & Board (per day) Rs.16,360 Rs.6,720 Rs.6,720 Rs.1,800 -Per Hospitalization -Pre-Hospitalization Sub Limit (Diagnosis, 30 Days 30 Days 30 Days 30 Days -Consultation, & Medicines) -Post-Hospitalization Sub Limit (Follow-Ups) 30 Days 30 Days 30 Days 30 Days -Daycare Surgeries & Specialized -Investigations In Outpatient Settings -Including but not limited to: -COVERED -Dialysis, Cataract Surgery, MRI, CT Scan, -Endoscopy, Thallium Scan, Angiography, Treatment -of Fractures, Local Road Ambulance for -Emergencies only, Emergency Dental Treatment -due to accidental injuries within 48 hours (for pain -relief only). - Plan A Presidential Layer - Plan B Associate Managers, Managers & Senior Managers - Plan C Executives, Senior Executives & Assistant Managers - Plan D Office Boys & Other Support Staff - Send us the cost estimate from the concerned doctor of the treatment with details, for prior -approval. Get the treatment and pay cash for the treatment. - Submit all original bills /supporting documents* including our approval letter with our claim form -for reimbursement, within 30 days of discharge from the hospital. - Settlement of claim will be done in line of prior approval given and other policy terms. -Supporting documents -Itemized hospital bill -Discharge card/clinical summary -Diagnostic reports -Prescriptions -Payment receipts - - -EOBI -Employee Old Age Benefit Income is a benefit provided to all full-time employees upon confirmation. -The sole purpose of EOBI is to provide compulsory social insurance to employees. It extends following -benefits to insured persons or their survivors: -• Old-Age Pension -• Survivor's Pension -• Invalidity Pension -• Old-Age Grant  -RATES: -• 1- Employer's Contribution @ 5 % of the worker's minimum wages (i.e. Rs. 8000) - Rs. 400/= Per -Month -• 2- Employee's Contribution @ 1 % of the worker's minimum wages (i.e. Rs.8000) - Rs. 80/= Per -Month -Provident Fund -Provident Fund is an investment/ saving fund contributed to - by employees & employers, out of which a -lump sum amount is provided to each employee on retirement. This benefit is provided to all full time, -confirmed employees. -Click Here To View PF Policies in Detail - - diff --git a/backup_2026-05-13_pre_intelligent_retrieval/floaitng-icon/KhloeSAC.lottie b/backup_2026-05-13_pre_intelligent_retrieval/floaitng-icon/KhloeSAC.lottie deleted file mode 100644 index dd4649d7db83bd2603fe9137d6e804d465753102..0000000000000000000000000000000000000000 --- a/backup_2026-05-13_pre_intelligent_retrieval/floaitng-icon/KhloeSAC.lottie +++ /dev/null @@ -1,3 +0,0 @@ -version https://git-lfs.github.com/spec/v1 -oid sha256:532a144eeea5a1a6e49ab22ae4c0dc4f94e12cf8540e3514f1bc676d929eeac2 -size 462684 diff --git a/backup_2026-05-13_pre_intelligent_retrieval/get_models.py b/backup_2026-05-13_pre_intelligent_retrieval/get_models.py deleted file mode 100644 index e7bcf221c211cef860ae9402c8e3adbaa648b025..0000000000000000000000000000000000000000 --- a/backup_2026-05-13_pre_intelligent_retrieval/get_models.py +++ /dev/null @@ -1,25 +0,0 @@ -import asyncio -import os -import httpx -from dotenv import load_dotenv - -load_dotenv() - -async def main(): - api_key = os.getenv("GROQ_API_KEY") - if not api_key: - print("No API Key") - return - url = "https://api.groq.com/openai/v1/models" - headers = {"Authorization": f"Bearer {api_key}"} - async with httpx.AsyncClient() as client: - resp = await client.get(url, headers=headers) - if resp.status_code == 200: - models = resp.json().get("data", []) - for m in models: - if "deepseek" in m["id"].lower() or "qwen" in m["id"].lower(): - print(m["id"]) - else: - print(resp.status_code, resp.text) - -asyncio.run(main()) diff --git a/backup_2026-05-13_pre_intelligent_retrieval/martech_sol_logo.jpg b/backup_2026-05-13_pre_intelligent_retrieval/martech_sol_logo.jpg deleted file mode 100644 index 4df90316265efa116ebc6a35a7a300c8515efdf1..0000000000000000000000000000000000000000 Binary files a/backup_2026-05-13_pre_intelligent_retrieval/martech_sol_logo.jpg and /dev/null differ diff --git a/backup_2026-05-13_pre_intelligent_retrieval/requirements.txt b/backup_2026-05-13_pre_intelligent_retrieval/requirements.txt deleted file mode 100644 index 88a16cc36cae5251255217fcc5504d57b676f9c3..0000000000000000000000000000000000000000 --- a/backup_2026-05-13_pre_intelligent_retrieval/requirements.txt +++ /dev/null @@ -1,14 +0,0 @@ -fastapi -uvicorn[standard] -faiss-cpu -sentence-transformers -pydantic-settings -pypdf -python-dotenv -numpy -httpx -gradio>=5.0.0 -python-multipart -pandas -openpyxl -rank_bm25 diff --git a/backup_2026-05-13_pre_intelligent_retrieval/static/addon.html b/backup_2026-05-13_pre_intelligent_retrieval/static/addon.html deleted file mode 100644 index 7d1cf1c953f2eef06ab8a7805aa69c18b79b3442..0000000000000000000000000000000000000000 --- a/backup_2026-05-13_pre_intelligent_retrieval/static/addon.html +++ /dev/null @@ -1,543 +0,0 @@ - - - - - -Martechsol Assistant Widget - - - - - - - -
-
- -
- - -
- - -
- -
-
-
- Martechsol Assistant -
- - -
-
Welcome! How can I help you today?
-
- -
- - -
-
-
- - - - - \ No newline at end of file diff --git a/backup_2026-05-13_pre_intelligent_retrieval/static/bot-icon.lottie b/backup_2026-05-13_pre_intelligent_retrieval/static/bot-icon.lottie deleted file mode 100644 index dd4649d7db83bd2603fe9137d6e804d465753102..0000000000000000000000000000000000000000 --- a/backup_2026-05-13_pre_intelligent_retrieval/static/bot-icon.lottie +++ /dev/null @@ -1,3 +0,0 @@ -version https://git-lfs.github.com/spec/v1 -oid sha256:532a144eeea5a1a6e49ab22ae4c0dc4f94e12cf8540e3514f1bc676d929eeac2 -size 462684 diff --git a/backup_2026-05-13_pre_intelligent_retrieval/static/chat-loader.js b/backup_2026-05-13_pre_intelligent_retrieval/static/chat-loader.js deleted file mode 100644 index ecce31c40c18beb75e41c98eed862d96984bb358..0000000000000000000000000000000000000000 --- a/backup_2026-05-13_pre_intelligent_retrieval/static/chat-loader.js +++ /dev/null @@ -1,48 +0,0 @@ -/** - * Martechsol Assistant Loader - * Usage: - */ -(function () { - const baseUrl = 'https://joedown11-chatrag.hf.space'; - - // Create container - const container = document.createElement('div'); - container.id = 'martech-chat-widget-container'; - document.body.appendChild(container); - - // Fetch the addon HTML from the dedicated /widget endpoint - fetch(`${baseUrl}/widget?v=${Date.now()}`) // Cache busting - .then(response => response.text()) - .then(html => { - // Create a temporary element to parse the HTML - const parser = new DOMParser(); - const doc = parser.parseFromString(html, 'text/html'); - - // 1. Extract and inject styles - doc.querySelectorAll('style').forEach(st => { - document.head.appendChild(st.cloneNode(true)); - }); - - // 2. Extract and inject body content - const wrapper = doc.querySelector('#martech-chat-wrapper'); - if (wrapper) { - container.innerHTML = wrapper.outerHTML; - } else { - // Fallback: just take the body content - container.innerHTML = doc.body.innerHTML; - } - - // 3. Extract and execute scripts (critical for logic) - doc.querySelectorAll('script').forEach(oldScript => { - const newScript = document.createElement('script'); - Array.from(oldScript.attributes).forEach(attr => { - newScript.setAttribute(attr.name, attr.value); - }); - newScript.innerHTML = oldScript.innerHTML; - document.body.appendChild(newScript); - }); - - // console.log("Martechsol Assistant Loaded Successfully"); - }) - .catch(err => console.error("Failed to load Martechsol Assistant:", err)); -})(); diff --git a/backup_2026-05-13_pre_intelligent_retrieval/test_groq.py b/backup_2026-05-13_pre_intelligent_retrieval/test_groq.py deleted file mode 100644 index 881464dbb8b59db2da7b164fa708b46835d7b2e8..0000000000000000000000000000000000000000 --- a/backup_2026-05-13_pre_intelligent_retrieval/test_groq.py +++ /dev/null @@ -1,26 +0,0 @@ -import asyncio -import os -import httpx -from dotenv import load_dotenv - -load_dotenv() - -async def main(): - api_key = os.getenv("GROQ_API_KEY") - url = "https://api.groq.com/openai/v1/chat/completions" - headers = { - "Authorization": f"Bearer {api_key}", - "Content-Type": "application/json" - } - payload = { - "model": "deepseek-r1-distill-llama-70b", - "temperature": 0.0, - "max_tokens": 1200, - "messages": [{"role": "system", "content": "hello"}, {"role": "user", "content": "test"}] - } - async with httpx.AsyncClient() as client: - resp = await client.post(url, headers=headers, json=payload) - print(resp.status_code) - print(resp.text) - -asyncio.run(main()) diff --git a/backup_2026-05-13_pre_intelligent_retrieval/wordpress code/WP_INTEGRATION_SCRIPT.txt b/backup_2026-05-13_pre_intelligent_retrieval/wordpress code/WP_INTEGRATION_SCRIPT.txt deleted file mode 100644 index f4d892fb2e96f15d663b36c219dcc345b9232dd8..0000000000000000000000000000000000000000 --- a/backup_2026-05-13_pre_intelligent_retrieval/wordpress code/WP_INTEGRATION_SCRIPT.txt +++ /dev/null @@ -1,12 +0,0 @@ - - - diff --git a/backup_2026-05-13_pre_intelligent_retrieval/wordpress code/addon.html b/backup_2026-05-13_pre_intelligent_retrieval/wordpress code/addon.html deleted file mode 100644 index aefe151d3d86fc85a597e5dcb1fe84c4f42d6c77..0000000000000000000000000000000000000000 --- a/backup_2026-05-13_pre_intelligent_retrieval/wordpress code/addon.html +++ /dev/null @@ -1,477 +0,0 @@ - - - - - -Martechsol Assistant Widget - - - - - - - -
-
- -
- - -
- - -
- -
-
-
- Martechsol Assistant -
- - -
-
Welcome! How can I help you today?
-
- -
- - -
-
-
- - - - - \ No newline at end of file diff --git a/backup_2026-05-13_pre_intelligent_retrieval/wordpress code/addon_backup.html b/backup_2026-05-13_pre_intelligent_retrieval/wordpress code/addon_backup.html deleted file mode 100644 index d1155a0d2593c40e1002009b2da9fc0b8ea7cb59..0000000000000000000000000000000000000000 --- a/backup_2026-05-13_pre_intelligent_retrieval/wordpress code/addon_backup.html +++ /dev/null @@ -1,175 +0,0 @@ - - - - - -
-
- -
- - -
- - -
- -
-
-
- Martechsol Assistant -
- -
-
- - \ No newline at end of file