{% extends "zerver/portico.html" %}

{% block title %}
<title>Tools and data sets | Zulip Dev</title>
{% endblock %}

{# Login page. #}
{% block portico_content %}

<div class="app flex full-page">
    <div id="devtools-page" class="markdown">
        <h1>Useful development URLs</h1>
        <p>
            Below is a list of useful tools and data sets available only in the Zulip
            development environment that are often useful when contributing to Zulip.
            Most of these require you to run a command to build/generate the relevant
            content. This table specifies which command to use to update the data served
            by each page (since several of these, like test coverage, require a special
            command to be run to generate the data). Make sure your development server is still running
            when you visit these!
        </p>
        <table class="table table-striped table-rounded table-bordered">
            <thead>
                <tr>
                    <th>URL</th>
                    <th>Command</th>
                    <th>Description</th>
                </tr>
            </thead>
            <tbody>
                <tr>
                    <td><a href="/coverage/index.html">/coverage/index.html</a></td>
                    <td><code>./tools/test-backend --coverage</code></td>
                    <td>Backend (Django) test coverage report</td>
                </tr>
                <tr>
                    <td><a href="/node-coverage/index.html">/node-coverage/index.html</a></td>
                    <td><code>./tools/test-js-with-node --coverage</code></td>
                    <td>Frontend (node) test coverage report</td>
                </tr>
                <tr>
                    <td><a href="/docs/index.html">/docs/index.html</a></td>
                    <td><code>./tools/build-docs</code></td>
                    <td>Developer documentation (ReadTheDocs) built locally</td>
                </tr>
                <tr>
                    <td><a href="/emails">/emails</a></td>
                    <td>View outgoing and example emails.</td>
                </tr>
                <tr>
                    <td><a href="/stats/realm/analytics/">/stats/realm/analytics/</a></td>
                    <td><code>./manage.py populate_analytics_db</code><br />
                        Run the command after changing analytics data population logic.
                    </td>
                    <td>View the /stats page with some pre-populated data</td>
                </tr>
                <tr>
                    <td><a href="/webpack/5xx.html">/webpack/5xx.html</a></td>
                    <td>None needed</td>
                    <td>Error 5xx page served by nginx (used when Django is totally broken)</td>
                </tr>
                <tr>
                    <td><a href="/errors/404">/errors/404</a></td>
                    <td>None needed</td>
                    <td>Error 404 page served by Django</td>
                </tr>
                <tr>
                    <td><a href="/errors/5xx">/errors/5xx</a></td>
                    <td>None needed</td>
                    <td>Error 5xx page served by Django</td>
                </tr>
                <tr>
                    <td><a href="/accounts/do_confirm/invalid">/accounts/do_confirm/invalid</a></td>
                    <td>None needed</td>
                    <td>Invalid confirmation link page</td>
                </tr>
                <tr>
                    <td><a href="/devtools/integrations">/devtools/integrations</a></td>
                    <td>None needed</td>
                    <td>Test incoming webhook integrations</td>
                </tr>
            </tbody>
        </table>
        <h2>Useful management commands</h2>
        <p>Development-specific <a href="https://zulip.readthedocs.io/en/latest/production/management-commands.html">management commands</a> live in <code>zilencer/management/commands</code>.  Highlights include:
            <ul>
                <li><code>./manage.py populate_db</code>: Rebuilds database. Has options to, for example, create 3K users for testing.</li>
                <li><code>./manage.py mark_all_messages_unread</code>: Useful for testing reading messages.</li>
                <li><code>./manage.py create_realm</code>: Add a new realm. Useful for testing onboarding.</li>
                <li><code>./manage.py create_user</code>: Add a new user. Useful for testing onboarding.</li>
                <li><code>./manage.py send_zulip_update_announcements</code>: Send <a href="https://zulip.com/help/configure-automated-notices#zulip-update-announcements">Zulip
                update notices</a> drafted in `zerver/lib/zulip_update_announcements.py`.</li>
                <li><code>./manage.py add_mock_conversation</code>: Add test messages, streams, images, emoji, etc.
                    into the dev environment. First edit zilencer/management/commands/add_mock_conversation.py
                    to add the data you're testing.
                </li>
            </ul>
        </p>
        <p>We also have
            <a href="https://zulip.readthedocs.io/en/latest/development/authentication.html">documentation on testing LDAP, Google &amp; GitHub authentication</a> in the development environment.
        </p>
        <h2>Connecting to the local PostgreSQL database</h2>
        <ul>
            <li>
                <code>./manage.py dbshell</code>: Connect to
                PostgreSQL database via your terminal.
            </li>
            <li>
                <code>provision</code> creates a <code>~/.pgpass</code> file,
                so <code>psql -U zulip -h localhost</code> works too.
            </li>
            <li>
                <p>
                    To connect using a graphical PostgreSQL client
                    like <a href="https://www.pgadmin.org/">pgAdmin</a>,
                    use the following credentials:
                </p>
                <ul>
                    <li>Host: 127.0.0.1 (don't use localhost)</li>
                    <li>Maintenance database: zulip</li>
                    <li>Username: zulip</li>
                    <li>password: stored as <code>local_database_password</code> in <code>zulip/zproject/dev-secrets.conf</code></li>
                </ul>
            </li>
        </ul>
        <h2>Development instructions for Help Center (Beta)</h2>
        <p>
            These commands are to run the project for an ongoing migration for our help center docs to use
            <a href="https://github.com/withastro/starlight">@astrojs/starlight</a>. You can track the
            progress for the project at <a href="https://github.com/zulip/zulip/issues/30450">#30450</a>.
        </p>
        <ul>
            <li>
                <code>./tools/build-help-center</code> will convert the existing help center MD files to MDX,
                and create a production build for the converted MDX files.
                <ul>
                    <li>
                        You can run <code>./tools/convert-help-center-docs-to-mdx</code> to convert the help
                        center MD files without the build step; see running a dev server that supports hot
                        reload below.
                    </li>
                    <li>
                        You can run <code>pnpm build</code> within the <code>help-beta</code> directory to run
                        the build step separately.
                    </li>
                </ul>
            </li>
            <li>
                <code>./tools/run-dev --help-center</code> will host the generated build on
                <code>/help-beta</code>. Note that search will work in the beta help center docs with
                this flag. For testing changes related to the migration of the help center docs, using the tool
                to build the beta help center docs and running the dev server with this flag should be fine.
            </li>
            <li>
                <code>./tools/run-dev --help-center-dev-server</code> will run a dev server at
                <code>/help-beta</code> that supports hot reload. Note that, with this flag, search will not work
                in the beta help center docs. This mode is useful when you are editing a help center file, and
                want to visualize the changes quickly in the beta help center documentation. You will need the
                converted MDX files to be already generated through
                <code>./tools/convert-help-center-docs-to-mdx</code> before you run the dev server with this flag.
            </li>
        </ul>
    </div>
</div>

{% endblock %}