Commit Graph

36032 Commits

Author SHA1 Message Date
Steve Howell 34191f2a56 generate_secrets: Avoid unnecessary settings import.
We try to avoid importing Django settings unless
we really need them, since we want this program
to run very quickly during `provision` (when
secrets have already been generated earlier).
2020-04-30 13:52:54 +00:00
Steve Howell 522ee7fd6b generate_secrets: Lazily import crypto module. 2020-04-30 13:49:44 +00:00
Steve Howell 66392afe15 generate_secrets: Extract random_token(). 2020-04-30 13:23:13 +00:00
Kartik Srivastava 7be52449a8 lib/bugdown/api_code_examples: Fix buggy rendering of print statement.
This fixes the buggy rendering of the print statement in the
python code examples.
2020-04-29 23:21:42 -07:00
Tim Abbott b246723d83 migrations: Add Message indexes on (recipient_id, subject).
Our previous set of indexes for the Message table did not contain
anything to optimize queries for all the messages in a topic in an
organization where the same topic name might appear in 10,000s of
messages in many streams.

We add two indexes here to support common queries
* A `(recipient_id, upper(subject), id)` index to support
  "Fetch all messages from a topic" queries.
* A `(recipient_id, subject, id)` index to support
  "Fetch all messages by topic"

We use the `DESC NULLS last` on both indexes because we almost always
want to query from the "Latest N messages" on a topic, not the
"Earliest N messages".

These indexes dramatically improve the performance of fetching topic
history (which remains not good enough in my opinion; we'll likely
need caching to make it nice), and more importantly make it possible
to check quickly which users have sent messages to a topic for the
"Topics I follow" feature.

Fixes part of #13726.
2020-04-29 23:11:52 -07:00
Tim Abbott 8bba5dc6b6 test_fixtures: Include LOCAL_DATABASE_PASSWORD in digest.
This ensures that if one deletes `zproject/dev-secrets.conf`, we end
up rebuilding the databases from scratch (which, critically, will
ensure the password that gets setup matches what's in the current
version of the configuration file).

This should address a category of issue we've had where deleting
`zproject/dev-secrets.conf` would result in provision failing.
2020-04-29 22:57:37 -07:00
Tim Abbott 34cd0d3530 provision: Avoid including all of dev-secrets.conf in digests.
This allows us to edit unrelated secrets without forcing a
reinitialization of RabbitMQ.
2020-04-29 22:53:49 -07:00
Ryan Rehman e1ee1adbd5 tests: Cleanup test_message_edit_notifications.
We add a `_send_and_update_message` function to
de-duplicate it's action.
2020-04-29 17:15:51 -07:00
vaibhavrajsingh2001 e3e7b0a6dd i18n: Add translation tags to digest emails. 2020-04-29 17:04:48 -07:00
sahil839 3b99e3440e actions: Avoid unnecessary fake email address changes.
The logic in do_set_realm_property would previously "change" the email
addrssees of every user in the realm, even if they hadn't actually
changed.

We fix this by skipping the logic when it's unnecessary.
2020-04-29 16:50:08 -07:00
sahil839 ce6ea76110 actions: Use bulk_update when changing email_address_visibility.
bulk_update is used to update the email of user_profile objects in
database when email_address_visibility is changed.

This helps resolve the problem of timeout errors in realms with large
number of users due to large number of database queries run in a
loop.

Since bulk_update doesn't flush caches, we need our own bit of code to
do that.

Fixes a part of #14600.
2020-04-29 16:45:11 -07:00
Pranav 069da46769 composebox: Make empty msg alert's cross prominent in night mode.
Increased opacity of the cross in night mode to make the cross easily
visible. Previously, low values of opacity caused the cross to be not
easily visible.

Fixes #14459.
2020-04-29 16:19:08 -07:00
Tim Abbott de22da768d api docs: Fix typo s/emoji_type/reaction_type. 2020-04-29 16:03:05 -07:00
Steve Howell aff58ed9d9 db tools: Simplify args for postgres-init-dev-db.
We now just have two modes for setting up a dev/test
database.  This makes it easy to see these things
side-by-side, when you're trying to understand how
the two different databases get built:

    # dev:
        USERNAME=zulip
        DBNAME=zulip
        STATUS_FILE_NAME=migration_status_dev

    # test:
        USERNAME=zulip_test
        DBNAME=zulip_test
        STATUS_FILE_NAME=migration_status_test

And then we make it more explicit the things that
are common between dev and test (which are
important things to understand when troubleshooting
provision-related glitches):

    SEARCH_PATH=zulip,public
    PASSWORD=$("$(dirname "$0")/../../scripts/get-django-setting" LOCAL_DATABASE_PASSWORD)
    DBNAME_BASE=${DBNAME}_base

We lose some "generality" here, but passing in arbitrary
combinations of username/dbname/status_file to the script
would cause chaos for our digest checks, and all the different
template/base databases could cause confusion too.
2020-04-29 12:30:49 -07:00
Steve Howell 2c63130195 api: Always return zulip_version/zulip_feature_level.
We no longer make these conditional to simplify writing clients;
there's no cost to including them, and a real cost to not doing so.
2020-04-29 11:58:22 -07:00
Tim Abbott 1b8ab92058 migrations: Mark several RunSQL migrations as elidable. 2020-04-29 11:02:34 -07:00
Aman Agrawal 6069d04696 ci: Save and restore zulip-emoji-caches on all jobs. 2020-04-29 10:54:53 -07:00
Aman Agrawal c2b8f8f7ed ci: Add run scripts to save and restore emoji cache. 2020-04-29 10:54:53 -07:00
Aman Agrawal 01b267dc0a ci: Remove old TODO statement. 2020-04-29 10:54:53 -07:00
Aman Agrawal 7e8ba97d85 ci: Add focal production install job. 2020-04-29 10:54:53 -07:00
Aman Agrawal 3442182d7b ci: Use absolute path to zulip folder.
Strangely focal resolves `~/zulip` as `/root/zulip`, this makes
sure we are in the correct folder.
2020-04-29 10:54:53 -07:00
Aman Agrawal 47bf111de8 migrations: Mark RunPython statements elidable.
This will make django automatically remove them when we run
squashmigrations. There are still some RunSQL statements which
we will have to take care of manually.
2020-04-29 10:41:20 -07:00
Steve Howell cf3abcedfd provision: Run configure_rabbitmq on clean checkouts.
We now just automatically run configure_rabbitmq any
time there's a clean checkout, or if an important
file has changed.

This allows us to bypass the step to import
SimpleQueueClient in certain places and reduce
some log spam.

See https://chat.zulip.org/#narrow/stream/3-backend/topic/new.20base.20dev.20droplet/near/864672
2020-04-29 13:18:54 -04:00
Jagan 805ac2475b settings: Redesign edit bot form.
This removes the weird edit-bot sidebar, replacing it with a modal,
matching our edit-user widget (and various similar ones).

Fixes #13644 by removing the buggy code.
2020-04-28 23:08:45 -07:00
Hashir Sarwar 0cbb778af4 ui_init: Fix play button position on video thumbnails.
This fixes a jQuery bug that broke the postion of play
button on video thumbnails particularly when YouTube
images fetched from oembed are displayed.
2020-04-28 22:41:00 -07:00
Hashir Sarwar 3a110bd647 actions: Remove redundant 'url_embed_preview_enabled' check.
We add URLs to the `links_for_embed set`, only when
the `url_embed_preview_enabled` flag is turned on.
So, it is sufficient to check if `links_for_embed`
is not empty.
2020-04-28 22:41:00 -07:00
Tim Abbott e8c57f2466 api docs: Document API level 2 change in custom emoji. 2020-04-28 22:23:25 -07:00
Tim Abbott cbe0723199 api docs: Add an API changelog page.
I imagine this can be improved in various ways, but I've initialized
this with all the **Changes** entries recorded in either zulip.yaml or
the rest of the API documentation, and I expect we'll be able to
iterate on this effectively.

It'll also be useful as a record of changes that we should remember to
document the API documentation as we document more endpoints that
currently don't discuss these issues.

While working on this, I fixed various issues where feature levels
could be mentioned or endpoints didn't properly document changes.
2020-04-28 22:23:25 -07:00
Mateusz Mandera f1ec02b40a auth: Add ExternalAuthResult to manage data in authentication flows.
This new type eliminates a bunch of messy code that previously
involved passing around long lists of mixed positional keyword and
arguments, instead using a consistent data object for communicating
about the state of an external authentication (constructed in
backends.py).

The result is a significantly more readable interface between
zproject/backends.py and zerver/views/auth.py, though likely more
could be done.

This has the side effect of renaming fields for internally passed
structures from name->full_name, next->redirect_to; this results in
most of the test codebase changes.

Modified by tabbott to add comments and collaboratively rewrite the
initialization logic.
2020-04-28 22:19:02 -07:00
Vishnu KS 3f69500765 billing: Rename downgrade_realm_for_deactivation to downgrade_now. 2020-04-28 22:05:49 -07:00
Vishnu KS aca80206df billing: Rename change_plan_at_end_of_cycle to change_plan_status. 2020-04-28 22:05:49 -07:00
akashaviator 28d58c848d api: Guess emoji_type when adding a reaction.
This changes add_reaction in zerver.views.reactions to allow
calling POST ../messages/{message_id}/reactions api endpoint with
emoji_name only, even in the case of a custom emoji.
2020-04-28 20:49:39 -07:00
Steve Howell cf78cb0d6e templates: Use `<hr>` and `<br>` consistently.
We now prevent these variations:

    * <hr/>
    * <hr />
    * <br/>
    * <br />

We could enforce similar consistency for other void
tags, if we wished, but these two are particularly
prevalent.
2020-04-28 17:05:48 -07:00
Tim Abbott 1be4cc5f17 openapi: Fix servers section to work in online editor. 2020-04-28 14:27:43 -07:00
Puneeth Chaganti 80e432b466 tools: Move screenshot captured message to Python. 2020-04-28 13:00:09 -07:00
Puneeth Chaganti 88c297dffe tools: Allow specifying more options when capturing screenshot.
The name and directory into which the screenshot should be captured can be
specified now, apart from specifying the fixture file to use.
2020-04-28 13:00:09 -07:00
Puneeth Chaganti 4e5c30e7fd tools: Pass image path as an argument to the screenshot tool.
Instead of figuring out the image path from the integration name in the
puppeteer script, we do it in the `generate-integration-docs-screenshot`
script and pass it as an argument to `message-screenshot.js`.
2020-04-28 13:00:09 -07:00
Puneeth Chaganti 39bea656f7 generate-integration-docs-screenshot: Remove call to sys.exit.
Preparatory refactor to allow calling the function in a loop, to generate
screenshots for all integrations.
2020-04-28 13:00:09 -07:00
Puneeth Chaganti ac7d8ce4b3 generate-integration-docs-screenshot: Notify on webhook's stream.
Use the stream specified by `stream_name` on the WebhookIntegration, instead
of using a common stream to send messages.
2020-04-28 13:00:09 -07:00
Puneeth Chaganti 844bca55f8 integrations: Remove unused Trello message screenshot. 2020-04-28 13:00:09 -07:00
Puneeth Chaganti 2c325e34ac integrations: Remove mutable default argument for config_options. 2020-04-28 13:00:09 -07:00
Puneeth Chaganti 39fe206289 integrations: Change legacy arg to a bool from Optional[bool]. 2020-04-28 13:00:09 -07:00
Puneeth Chaganti f280b64caa integrations: Remove hasattr check for stream_name on Integrations.
`stream_name` is always set on `Integration`. If it is not explicitly
specified, it is set to the name of the integration.
2020-04-28 13:00:09 -07:00
orientor 4d8c988ef2 openapi: Use description markdown for rendering endpoint descriptions.
Firstly, change endpoint descriptions in zulip.yaml so that they
match their counterpart in the api docs. Then edit the api docs
so that they use api description markdown extension for displaying
endpoint description.
2020-04-28 12:57:19 -07:00
orientor 64c6bab276 openapi: Create markdown extension for rendering endpoint descriptions.
Add function in openapi.py to access endpoint descriptions written
in zulip.yaml. Use this function for creating a markdown extension
for rendering endpoint descriptions written in zulip.yaml.

We use this extension for a single endpoint to get test coverage.
2020-04-28 12:57:19 -07:00
Aman Agrawal cfe427b3f7 narrow: bugfix: Update stream list height after rendering completes.
When switching from Private Messages narrow to
All messages narrow, stream list max-height was not
correctly updated. Stream list max-height was calculated
 before new height were updated by browser for
All message narrow.

Inshort:
Stream list max-height was being updated before the browser could
render height for `#global_filters`. Calling resize after narrow
completes removes this issue.
2020-04-28 12:32:40 -07:00
arpit551 a6df0b94dd doc: Updated docs for the job xenial-legacy. 2020-04-28 11:28:07 -07:00
Tim Abbott 28cb0aa81a docs: Use consistent spelling of CircleCI. 2020-04-28 11:26:58 -07:00
arpit551 16d41e59b5 doc: Update documentation replacing travis with CircleCI. 2020-04-28 11:24:54 -07:00
arpit551 7f769512aa travis: Remove Travis unwanted code.
Since in travis we don't have root access so we used to add different
srv path. As now we shifted our production suites to Circle CI
we don't need that code so removed it.

Also we used a hacky code in commit-lint-message for travis which is
now of no use.
2020-04-28 11:11:23 -07:00