Commit Graph

8385 Commits

Author SHA1 Message Date
Tim Abbott 83300c329b register: Fix period at end of heading. 2018-08-09 17:17:45 -07:00
Shubham Padia bdaff17e28 narrow: Use is_private flag index for `is:private`.
This implements a significant performance optimization for users
clicking the `Private messages` narrow in the Zulip UI, especially for
those users who do not have 50 recent private messages in an
organization with a lot of stream message traffic (because then
previously, postgres needed to scan through a huge amount of history
to find enough private messages).

The database index powering it can also support many other queries we
might want to do in the future to support "recent conversations" type
features.

Fixes #6896.
2018-08-09 16:10:01 -07:00
Shubham Padia 40fdc4724b migrations: Set initial value for is_private flags.
This initializes the is_private flag to have the correct value for
historical messages.
2018-08-09 16:09:44 -07:00
Shubham Padia 9dcac3479e actions.py: Set is_private flag in do_send_messages. 2018-08-09 16:08:04 -07:00
Shubham Padia a524d425ad actions.py: Block client interaction with flags in the NON_API_FLAGS.
Raise error if flag is present in NON_API_FLAGS or is not present in
UserMessage.flags.
2018-08-09 16:08:03 -07:00
Tim Abbott fe9eeecda1 models: Do not leak 'active_mobile_push_notification' flag to API.
The reasoning here is similar to `is_private`; this flag is only used
for internal accounting inside the Zulip server.
2018-08-09 16:03:30 -07:00
Shubham Padia 2f895afdaa models: Do not leak `is_private` UserMessage flag to the API.
See the comment for why this is correct; basically, this flag is used
only for internal accounting, and would only confuse API clients.
2018-08-09 16:03:30 -07:00
Tim Abbott 25b646feaf decorator: Fix error message for accessing deactivated accounts.
The previous message was potentially a lot more ambiguous about
whether this was something about presence.  "Deactivated" makes it
explicit that some action was taken to deactivate the account.
2018-08-09 15:58:31 -07:00
Tim Abbott a124db5f7b decorator: Check for deactivated realm before deactivated user.
Structurally, this is the right order-of-access for data.
2018-08-09 15:57:18 -07:00
Tim Abbott 4c4b6d105e import: Fix re-rendering of markdown for Zulip->Zulip imports.
The code added in 26300110ca was only
needed for importing data from Slack, Gitter, or another tool which
doesn't use Zulip's markdown format.
2018-08-09 15:15:50 -07:00
Rhea Parekh 26300110ca import: Fix rendered_content in imported messages.
After the messages have been imported, set the rendered_content of the
messages instead of leaving its value to be 'None'.

This is important to ensure that:
(1) Performance for users is good after completing the import.
(2) The database's full-text indexes have all of the imported messages
(which only happens properly when Message rows have their
rendered_content field edited).

Fixes #9168.
2018-08-09 15:12:53 -07:00
Eeshan Garg 01bd55bbcb render_markdown_path: Refactor pure_markdown rendering logic.
The changes in this commit were suggested by Tim Abbott in #10252.
2018-08-09 14:34:52 -07:00
Eeshan Garg 9e2a369e42 render_markdown_path: Add test for templates with absolute paths.
In certain cases we have to load a template directly because it
isn't in Jinja2's recognized template directories. This commit
adds a test to make sure that absolute paths are recognized
if they are pure Markdown files.
2018-08-09 14:34:52 -07:00
Shubham Padia d409555b2f auth: Add function for generating test ldap_dir to backends.py.
Generates ldap_dir based on the mode and the no. of extra users.
It supports three modes, 'a', 'b' and 'c', description for which
can be found in prod_settings_templates.py.
2018-08-09 13:46:44 -07:00
Eeshan Garg 0a99fa2fd6 render_markdown_path: Handle absolute template paths properly.
One disadvantage of relying on Jinja2 to load all templates is that it
only searches a finite set of pre-configured template directories.
Unfortunately, that breaks when someone tries to enable a custom
privacy or terms page and has the corresponding template in a
directory outside of Jinja2's recognized directories (for instance, it
won't find `/etc/zulip/terms.md`, the recommended path).

This commit makes it so that render_markdown_path can be more
sensible about pure Markdown files and load templates with
absolute paths directly without relying on Jinja2, if need be.
2018-08-09 08:43:03 -07:00
Tim Abbott 3e885e52b2 i18n: Fix hard-to-translate notification bot string. 2018-08-09 08:37:23 -07:00
Steve Howell dc70bc1d3f tests: Make retention tests less time-sensitive.
We now update all test messages to have a pub_date
of "now" in the setUp() function in TestRetentionLib.

We've seen tests flake on query counts before this
patch.  It's not certain that the test flaked due
to time-related glitches, but it seems the most
plausible explanation.
2018-08-09 06:13:40 -04:00
Steve Howell 2e42bdd7c3 Remove "/stats" command for now.
The "/stats" command doesn't actually do anything
interesting yet, and it also writes to the message
feed instead of replying directly to the user.

The history of this command was that it was
written during a PyCon sprint.  It was mainly intended
as an example for subsequent slash commands.  The
ones we built after "/stats" have sort of outgrown
"/stats" and don't follow the original structure
for "/stats".  (The "/day", "/ping", and "/settings"
commands were built shortly after.)j

We probably want to ressurect "/stats" fairly soon,
after figuring out some useful stats and refining
the UI.

As you can see from this commit, resurrecting the
code here shouldn't be too difficult, but it
may actually be pretty rare that we just translate
slash commands into fleshed out messages.
2018-08-08 16:49:27 -07:00
Yago González 298aa0fcbf mobile: Make otp_encrypt_api_key accept API keys.
Since otp_encrypt_api_key only encrypts API keys, it doesn't require
access to the full UserProfile object to work properly. Now the
parameter it accepts is just the API key.

This is preparatory refactoring for removing the api_key field on
UserProfile.
2018-08-08 16:45:40 -07:00
Yago González 6a192ac84c utils: Move random API key generator as generate_api_key.
random_api_key, the function we use to generate random tokens for API
keys, has been moved to zerver/lib/utils.py because it's used in more
parts of the codebase (apart from user creation), and having it in
zerver/lib/create_user.py was prone to cyclic dependencies.

The function has also been renamed to generate_api_key to have an
imperative name, that makes clearer what it does.
2018-08-08 16:45:25 -07:00
Yago González f6219745de users: Get all API keys via wrapper method.
Now reading API keys from a user is done with the get_api_key wrapper
method, rather than directly fetching it from the user object.

Also, every place where an action should be done for each API key is now
using get_all_api_keys. This method returns for the moment a single-item
list, containing the specified user's API key.

This commit is the first step towards allowing users have multiple API
keys.
2018-08-08 16:35:17 -07:00
Yago González 13b9dd33fa test_decorators: Explain typo-looking expression.
The validate_api_key sentence may look a bit confusing since we are
using webhook_bot's email address but default_bot's API key.

At first sight, and without any context on these tests, it may look like
that's just a typo, but we do want it to be like it is right now because
that way the API key used doesn't correspond to the provided email
address (triggering some untested parts of our backend logic).
2018-08-08 16:35:17 -07:00
Harshit Bansal 9057e543ac emoji: Disable support for letting users switch emojisets.
Due to copyright issues with potentially displaying Apple emojisets on
non-apple devices, as well as iamcal dropping support for the emojione
emojiset (see https://github.com/iamcal/emoji-data/pull/142), we are
dropping (perhaps temporarily) support for allowing users to switch
emojisets in Zulip.

This commit just hides the feature from the user but leaves most of
the infrastructure in place so that in the future if we decide to
re-enable the support we will not need to redo the infrastructure work
(some JS-side code is deleted, mostly because we'll want to re-add the
feature using the do_settings_change infrastructure anyway).

The most likely emoji set to add is the legacy "blobs" Google emoji
set, since it seems popular with some users.

Tweaked by tabbott to remove some additional JS code and update the
changelog.
2018-08-08 15:48:04 -07:00
Tim Abbott de45853c99 OpenAPI: Import yamole inside a function for performance.
This saves about 1% of the runtime of `manage.py showmigrations`
2018-08-08 14:19:42 -07:00
Tim Abbott 4d03c15848 url_preview: Don't import beautifulsoup at import time.
This is a small performance optimization to Django startup, in line
with other recent commits.
2018-08-08 14:19:42 -07:00
Tim Abbott 9941b408a0 email_log: Only import Django test client when using it.
Importing the Django test client is somewhat expensive, and we only
use it within one view function that's not used in production.  So
there's a significant startup-time performance optimization in doing
an import inside the view code.
2018-08-08 14:19:42 -07:00
Tim Abbott 38afeb7ac2 bugdown: Lazily import python-twitter.
python-twitter was consuming a significant amount of import time.
However, this commit seems to not save any time at all, probably
because its recursive dependencies are imported elsewhere in Zulip.
2018-08-08 14:19:42 -07:00
Shubham Dhama 4d07e485ff test_event_queue: Make subscription management code readable.
This test refactor makes the subscription/stream settings changes use standard
APIs and thus be easier to follow (and more robust to subtle re-fetching bugs).

This is a follow-up to #9181.
2018-08-08 12:39:19 -07:00
Cynthia Lin f68bb8132a zcommand: Fix broken switch mode message.
Instead of displaying the zcommand to switch to the other mode, the
message displayed the zcommand that the user had just entered.
2018-08-08 11:29:49 -07:00
Shubham Padia 17d81d46dc Realm filters: Allow `#`(hash) based URLs in filter patterns.
Fixes #9817.
Use case: When people try to add a filter to a site that uses
a hash based router, which makes the URL like:
`http://some-site.domain.com/#/c/$(id)s`.
2018-08-08 11:06:33 -07:00
Yago González 5566646c45 user_groups: Handle renaming to existing names.
Renaming a user group to a name shared by other group wasn't a scenario
handled by the backend, and the server errored whenever this was
attempted.

Now a json_error is returned, letting the user know that a user group
with that name already exists.
2018-08-08 11:03:47 -07:00
Yago González 14bc5c7d5c messages: Don't require "anchor" when "use_first_unread_anchor" is set.
The use_first_unread_anchor parameter allows automatically setting the
anchor to the first message that hasn't been read in this narrow.
Therefore it isn't necessary to specify an anchor when this parameter is
enabled.

Note from Tim: Arguably, we should think about making
`use_first_unread_anchor` the default behavior when anchor is
unspecified, but that's for later consideration.
2018-08-08 10:30:08 -07:00
Yago González f63b7ada60 api docs: Create the NonExistingStreamError schema definition. 2018-08-08 09:29:27 -07:00
Yago González 586e013069 api docs: Specify array types. 2018-08-08 09:29:27 -07:00
Yago González d45fac0327 api docs: Minor formatting improvements. 2018-08-08 09:29:27 -07:00
Yago González 8d0cf3ebe5 api docs: Migrate POST /user_uploads to OpenAPI. 2018-08-08 09:29:27 -07:00
Yago González 119b3c0bc4 api docs: Migrate DELETE /users/me/subscriptions to OpenAPI. 2018-08-08 09:29:27 -07:00
Yago González 14c9277095 api docs: Migrate POST /users to OpenAPI. 2018-08-08 09:29:27 -07:00
Yago González e1662024d1 api docs: Load the OpenAPI file only when needed.
We found out in #9953 that, appparently, loading the OpenAPI file was
taking abut a 5% of the Zulip server startup time.

Since in many cases (especially in development) having the file loaded
won't be necessary at all, we read it on the first time data from the
OpenAPI spec is needed.

Tweaked by tabbott to add a test.
2018-08-08 09:00:28 -07:00
Yago González 25d2efb9ca api docs: Live reload the OpenAPI spec on update.
Automatically detect if the OpenAPI spec file has been modified since
the last time it was loaded into memory, and if it has, automatically
reload it to have the latest version.

This feature is designed with development environments in mind. The main
benefit is being able to see the changes made to the OpenAPI document
without needing to restart the development server, which is tedious and
slows the documentation workflow down.
2018-08-08 08:54:25 -07:00
Rhea Parekh 18a4904437 import: Move 'build_attachment' to import_util. 2018-08-07 16:45:42 -07:00
Rhea Parekh b6ccc0bc52 import: Move 'build_defaultstream' to import_util. 2018-08-07 16:45:42 -07:00
Rhea Parekh bee3964f14 import: Move 'build_usermessages' to import_util. 2018-08-07 16:45:42 -07:00
Rhea Parekh eefe7cccd2 import: Move 'process_uploads' and 'process_emojis' to import_util. 2018-08-07 16:45:42 -07:00
Rhea Parekh 30cc7354eb import: Move 'process_avatars' to import_util. 2018-08-07 16:45:40 -07:00
Rhea Parekh 87cc1a6280 import: Move 'build_subscription' and 'build_recipient' to import_util. 2018-08-07 16:35:56 -07:00
Rhea Parekh a516f80646 import: Move 'build_avatar' to import_util. 2018-08-07 16:35:56 -07:00
Rhea Parekh 1117455a90 import: Move 'ZerverFieldsT' and 'build_zerver_realm' to import_util. 2018-08-07 16:35:56 -07:00
Yashashvi Dave 290388e5e0 stream settings: Fix bug in UI when last user unsubscribe private stream.
When last user(only in case of admin) unsubscribe from private stream,
stream page doesn't get updated. Cause we delete the private stream
as soon as last user unsubscribe from stream.
So `sub` get undefined in frontend, cause that stream is deleted
before unsubscribe-user-from-stream event is received.

Fix this by changing order of events sent to frontend. Event
`subscription: remove` should be sent before `stream: delete` event
from backend.
2018-08-07 13:30:53 -07:00
Yashashvi Dave 6e136be975 default stream: Allows admins to remove any default stream.
This fixes a bug where administrators couldn't remove private
unsubscribed streams from the "default streams" list, because
access_stream_by_name didn't give them access to the stream object.
2018-08-07 13:28:30 -07:00