Commit Graph

3597 Commits

Author SHA1 Message Date
Yago González 7d8e058e39 api docs: Implement an exception list for schema validation. 2018-08-02 15:54:41 -07:00
Tim Abbott e7c7211c30 mypy: Fix type of messages in do_update_message_flags.
Ever since we moved the stream/everything cases to separate functions,
the messages argument has actually been required.
2018-08-01 17:37:16 -07:00
Tim Abbott 23246ff816 do_update_pointer: Switch to using where_unread().
This produces a more efficient database query (just because postgres
doesn't use the right index by default the other way).
2018-08-01 16:51:56 -07:00
Tim Abbott c775be8ea4 do_mark_stream_messages_as_read: Accept a Client object.
We also fix an incorrect Optional in the type annotations.
2018-08-01 16:49:57 -07:00
Tim Abbott 6e55342e21 bulk_remove_subscriptions: Pass client object in.
We need the client object to pass on to do_mark_stream_as_read.
2018-08-01 16:48:31 -07:00
Tim Abbott c60f197fde do_update_message_flags: Accept a Client object.
This is important for upcoming logging changes.
2018-08-01 16:40:58 -07:00
Tim Abbott 0e44010976 do_mark_all_as_read: Accept a client object.
This is needed for upcoming logging changes.
2018-08-01 16:40:15 -07:00
Tim Abbott 5f0519dfb4 do_update_pointer: Pass client object from callers.
We also fix an unused import.

This is needed for upcoming logging changes.
2018-08-01 16:40:15 -07:00
Tim Abbott 54d558b128 management: Add library for getting a client object.
This is to be used in some analytics features we're adding in the near
future.
2018-08-01 16:40:15 -07:00
Tim Abbott e9f4d9db2b push_notifications: Fix interface for handle_remove_push_notification.
This really should just accept a message ID.
2018-08-01 16:36:42 -07:00
Kunal Gupta bc43eefbfb notifications: Add function for cancelling GCM notifications.
This adds a new function called handle_remove_push_notification in
zerver/lib/push_notifications.py which requires user_profile id and
the message id which has to be removed in the function.

For now, the function only supports GCM (and is mostly there for
prototyping).

The payload which is being delivered needs to contain the narrow
information and the content of the message.
2018-08-01 15:59:04 -07:00
Tim Abbott 5f8d193bb7 notifications: Include realm_uri in push notifications.
This should make it much simpler for the mobile apps to line up the
data from server_settings against the data in the notifications.

Addresses part of #10094.
2018-08-01 15:46:15 -07:00
Tim Abbott 58ee3fa8c4 page_params: Include avatars and similar data in cross-realm bots.
This ensures that the format of this data structures matches that for
in-realm bots in the main users data structure (including avatars,
etc.).

Fixes #10138.
2018-08-01 15:09:11 -07:00
Tim Abbott ccba1e7c0e cache: Limit which realms we access when filling caches.
For realms that don't have any presence-active users, we know for a
fact that there aren't any active clients that will be reloading just
after the server restarts, so we can skip filling the cache with data
related to that realm.

For zulipchat.com, this results in a significant performance
optimization for the recipient and stream caches, and a moderate
performance improvement for the user caches as well.
2018-08-01 14:22:49 -07:00
Tim Abbott e6abc6e0bd cache: Only ensure we fill recipient caches for Recipient.STREAM.
Private messages make up the bulk of Recipient objects.  While private
messages are ~50% of messages, if you weight by messages received
(which is what is important for message-loading performance), it's
pretty strongly balanced towards stream messages.
2018-08-01 13:09:59 -07:00
Tim Abbott 463a348a86 cache: Don't pre-populate low-traffic user caches.
We only really need to pre-populate the caches used for (1) mobile
authentication and (2) most other user lookups.
2018-08-01 13:09:34 -07:00
Tim Abbott c42302e47b restart-server: Optimize prepopulating user cache.
We don't need to include long-term idle or other inactive users here,
since fetching them consumed to vast majority of the time.

(On chat.zulip.org, this decreased the runtime for populating the user
cache by 5x, removing only users we're unlikely to need to access).
2018-08-01 12:54:06 -07:00
Tim Abbott f10e006135 message: Bump MAX_UNREAD_MESSAGES to 50000.
This doesn't seem to have a huge performance downside (less than 1s
extra time for loading / on chat.zulip.org), and it means the
possibility of users having so many unreads that we get weird/buggy
behavior is much more unlikely to exist.

We'll still want a better experience for users who somehow go over
this limit, but it can be pretty firmly "you need to go mark some
things as read".
2018-08-01 12:02:54 -07:00
Rhea Parekh ee37866687 import: Add gitter import file in zerver/data_import directory. 2018-08-01 11:52:14 -07:00
Rhea Parekh b8e1e8b31d import: Add slack import files in zerver/data_import directory. 2018-08-01 11:52:14 -07:00
Vishnu Ks 6b3706494c notifications: Pass realm_creation argument to enqueue_welcome_emails. 2018-08-01 11:29:34 -07:00
Roman Godov 5e70577f84 models: Rename Realm.show_digest_email field.
This renames Realm.show_digest_email field to
digest_emails_enabled, for greater clarity as to what it does
just from seeing the setting name, without having to look it up.

Fixes part of #10042.
2018-08-01 11:05:58 -07:00
Tim Abbott 6f7e12ea19 docs: Add subsystem documentation for caching. 2018-07-31 17:00:45 -07:00
Tim Abbott 1b2a26ca83 events: Fix missing empty custom profile data dict for new users.
We were getting event-handling exceptions in JS in production if a new
user was created and then went and set a custom profile field, because
there was no `.profile_data` on their user object.  We were able to
trace the issue down to the fact that our events didn't include that
field when creating a new user.
2018-07-31 11:08:11 -07:00
Roman Godov c0806917ec models: Rename Realm.restricted_to_domain field.
This renames Realm.restricted_to_domain field to
emails_restricted_to_domains, for greater clarity as to what it does
just from seeing the setting name, without having to look it up.

Fixes part of #10042.
2018-07-31 09:28:33 -07:00
Cynthia Lin 29442ffb93 zcommand: Add light/dark mode command aliases for day/night mode.
Fixes #10095.
2018-07-31 07:12:31 -04:00
Tim Abbott e04156eef3 export: Fix error messages for stream list mismatches.
The previous error messages for this were written for a tool only to
be used by a couple people, and didn't make clear what potential
causes were.  Tweak these to provide greater clarity about what's
going on.

The main cause of these errors appearing in practice was fixed in
7ea5987e5d, but nothing strongly
prevents a similar issue from being introduced in the future.

Fixes #10078.
2018-07-30 22:32:26 -07:00
Tim Abbott 6317064210 unminify: Fix source map extraction for hash-named files.
Apparently, our old unminify logic relied on the fact that the
filenames displayed in tracebacks were of the form "app.js" (and the
`app.js` copy of the source map in the appropriate
`/home/zulip/deployments/`).  The correct behavior is to just look up
the source map for the appropriate hash-named
`app.a40806b10565c1dee5bf.js` type file.

We fix this with a few small tweaks to the regular expressions.  I
wish this file had reasonable unit tests.
2018-07-30 22:09:37 -07:00
Tim Abbott aa5959396d docs: Add some basic subsystem documentation for thumbnailing. 2018-07-30 13:20:54 -07:00
Aditya Bansal 77651ece39 thumbnails: Rename size value 'original' to 'full'. 2018-07-30 13:00:23 -07:00
Aditya Bansal 5b5d8bb310 thumbnails: Rename data-original to data-src-fullsize. 2018-07-30 13:00:23 -07:00
Tim Abbott 02ae71f27f api: Stop using API keys for Django->Tornado authentication.
As part of our effort to change the data model away from each user
having a single API key, we're eliminating the couple requests that
were made from Django to Tornado (as part of a /register or home
request) where we used the user's API key grabbed from the database
for authentication.

Instead, we use the (already existing) internal_notify_view
authentication mechanism, which uses the SHARED_SECRET setting for
security, for these requests, and just fetch the user object using
get_user_profile_by_id directly.

Tweaked by Yago to include the new /api/v1/events/internal endpoint in
the exempt_patterns list in test_helpers, since it's an endpoint we call
through Tornado. Also added a couple missing return type annotations.
2018-07-30 12:28:31 -07:00
Shubham Dhama 499a70b01e notifications: Fix leaking of private stream msgs sent before subscribing.
Fixes: #9834.
2018-07-28 15:13:08 -07:00
Shubham Dhama c5d8fdf68c message: Add function to check message access in bulk.
This effectively just calls access_message in a loop.
2018-07-28 15:12:55 -07:00
Tim Abbott 6bbffe0e2e notifications: Extract zerver/lib/url_encoding.py.
Extracting this helper library will help us avoid an import loop
between notifications.py and message.py (with bugdown in between).

But in addition to that, it's a more natural model, since some of the
uses for these functions weren't part of the notifications code
anyway.
2018-07-28 15:12:55 -07:00
Shubham Dhama 605a6ed3d8 message: Extract function for message access by a user. 2018-07-28 14:58:36 -07:00
Shubham Dhama 9cf67a5e71 models: Add function to get UserMessage from message id. 2018-07-28 14:58:36 -07:00
Tim Abbott 38dd9e49de import_realm: Add comments for update_model_ids.
This basically explains why in these cases we delay doing
bulk_import_model.
2018-07-26 16:13:14 -07:00
Rhea Parekh fca6bc91aa import: Add 'get_db_table' function.
Implement this function in 'bulk_import_model'
and 'update_model_ids'.

This lets us save on redundant-feeling arguments in these
frequently-called helper functions.
2018-07-26 16:07:31 -07:00
Roman Godov 064fca5b76 import: Fix use of unused Subscription.notifications.
This code should have been interacting with the specific sub-fields
for notifications.
2018-07-26 15:53:29 -07:00
Yago González db81726bcb api docs: Use method from our Python bindings in the example.
One of the code examples for GET /users was using the raw call_endpoint
method from our Python bindings, rather than get_members, which has been
specifically designed to interact with this endpoint.

Now all the examples here use the appropiate method.
2018-07-26 15:34:31 -07:00
Yago González 40a21f6077 api docs: Migrate GET /users to OpenAPI. 2018-07-26 15:34:31 -07:00
Yago González 7c50f6cdd4 api docs: Migrate POST /messages/render to OpenAPI. 2018-07-26 15:34:31 -07:00
Yago González 57c2d8c72a api docs: Migrate GET /get_stream_id to OpenAPI. 2018-07-26 15:34:31 -07:00
Yago González 679319a735 api docs: Migrate GET /streams to OpenAPI. 2018-07-26 15:34:31 -07:00
Yago González 95ce311686 api docs: Display the default value of parameters when present.
Whenever a parameter for an endpoint in our REST API has a default
value, it is displayed under the "Description" section of the
arguments table in the docs.

This way, we don't need to explicitly indicate the default values in the
description, thus avoiding duplicate information in the OpenAPI source.
2018-07-26 15:34:31 -07:00
Yago González 7ddc6e8d10 api docs: Migrate POST /messages to OpenAPI. 2018-07-26 10:24:01 -07:00
Yago González cb24756edc api docs: Merge docs for sending stream and private messages. 2018-07-26 10:24:01 -07:00
Shubham Dhama 50ed378dd6 bugdown: Improve exception handler for BugdownRenderingException.
This has the benefit that we now get the usual data about the
user/request/etc. in error emails related to bugdown exceptions;
previously we were just getting the traceback in the emails (since our
`mail_admins` template was very simple) and no other debugging
details.

Comments tweaked by tabbott to help make clear exactly what's going on
here, since it's a little subtle and a little hacky.

Fixes #8843.
2018-07-26 09:27:55 -07:00
Shubham Dhama a7e913894c bugdown: Add bugdown_logger for logging. 2018-07-26 09:27:55 -07:00