Commit Graph

229 Commits

Author SHA1 Message Date
Vaibhav 61371cbe9a user_info: Add `bot_owner_id` to user info dataset.
Modifies the dict with the user info to include the key `bot_owner_id`
so it can be displayed in the user info popover.

Tests concerned with changing bot owner have been modified to have
number of events=2 because while updating the bot info, two events
are fired -- updating the `realm_bot` and `realm_user` since the
key `bot_owner_id` is a part of realm user info.
2019-05-28 12:41:52 -07:00
Tim Abbott 07856ad648 api: Override client_gravatar for EMAIL_ADDRESS_VISIBILITY_ADMINS.
Clients won't have access to user email addresses, and thus won't be
able to compute gravatars.

The tests for this are a bit messy, in large part because our tests
for get_events call subsections of it, rather than the main function.
2019-05-20 18:13:31 -07:00
Harshit Bansal b553507412 subscriptions: Migrate notification setting defaults model.
This commit migrates the Subscription's notification fields from a
BooleanField to a NullBooleanField where a value of None means to
inherit the value from user's profile.

Also includes a migrations to set the corresponding settings to None
if they match the user profile's values. This migration helps us in
getting rid of the weird "Apply to all" widget that we offered on
subscription settings page.

The mobile apps can't handle None appearing as the stream-level
notification settings, so for backwards-compatibility we arrange to
only send True/False to the mobile apps by applying those defaults
server-side.  We introduce a notification_settings_null value within a
client_capabilities structure that newer versions of the mobile apps
can use to request the new model.

This mobile compatibility code is pretty effectively tested by the
existing test_events tests for the subscriptions subsystem.
2019-05-08 17:45:10 -07:00
David Wood 34d810aac3 settings: Migrate to create_stream_policy structure.
This commit replaces the `create_stream_by_admins_only` setting with a
new `create_stream_policy` setting, which mirroring the structure of
the existing `invite_to_stream_policy`.

This is important preparation for migrating the waiting period feature
to be its own independent setting.

Fixes #12236.
2019-05-06 16:27:55 -07:00
David Wood 272ed90685 settings: Create an explicit invite_to_stream_policy setting.
This commit creates a new organization setting that determines whether
a user can invite other users to streams. Previously this was linked
to the waiting period threshold, but this was both not documented and
overly limiting.

With significant tweaks by tabbott to change the database model to not
involve two threshhold fields, edit the tests, etc.

This requires follow-up work to make the create stream policy setting
work how this code implies it should.

Fixes #12042.
2019-04-29 17:11:28 -07:00
Puneeth Chaganti b0f485e95f org settings: Allow admins to enable or disable digest emails. 2019-04-27 15:20:41 -07:00
Wyatt Hoodes d4715f23d7 public_export: Add backend API endpoint for triggering export.
An endpoint was created in zerver/views.  Basic rate-limiting was
implemented using RealmAuditLog.  The idea here is to simply log each
export event as a realm_exported event.  The number of events
occurring in the time delta is checked to ensure that the weekly
limit is not exceeded.

The event is published to the 'deferred_work' queue processor to
prevent the export process from being killed after 60s.

Upon completion of the export the realm admin(s) are notified.
2019-04-26 17:24:29 -07:00
Tim Abbott ddf26a6ced test_events: Post-process state before comparisons in do_test.
This is important because upcoming features will include slightly more
complex logic in post_process_state that we'd ideally like to be
included in what this suite tests.

This requires a few related changes:

* A small change to post_process_state to sort the realm_users objects
  by user_id to ensure those data structures are stable.

* Improvements to the logic for checking if the initial state has
  changed to use match_states for better output.
2019-04-08 19:42:34 -07:00
Tim Abbott 4c3c669b41 mobile: Add support for recent_private_conversations API.
This adds experimental support in /register for sending key
statistical data on the last 1000 private messages that the user is a
participant in.  Because it's experimental, we require developers to
request it explicitly in production (we don't use these data yet in
the webapp, and it likely carries some perf cost).

We expect this to be extremely helpful in initializing the mobile app
user experience for showing recent private message conversations.

See the code comments, but this has been heavily optimized to be very
efficient and do all the filtering work at the database layer so that
we minimize network transit with the database.

Fixes #11944.
2019-04-04 16:47:43 -07:00
Tim Abbott 32c50a99fd events: Pass a Realm to get_raw_user_data.
This will be helpful for checking realm settings in upcoming commits.
2019-04-04 13:05:35 -07:00
Puneeth Chaganti 52afbe5e8d soft-deactivation: Rename maybe_catch_up_soft_deactivated_user.
Rename `maybe_catch_up_soft_deactivated_user` to
`reactivate_user_if_soft_deactivated`.
2019-03-13 17:16:22 -07:00
Tim Abbott 662fa037f2 events: Don't assume that streams/subscriptions were requested.
This fixes a bug in our race condition handling in the case that
streams and/or subscriptions register content was not requested by the
client.
2019-03-13 14:12:59 -07:00
Challa Venkata Raghava Reddy b69aec2dbc streams: Add first_message_id tracking first message in stream.
This field is primarily intended to support avoiding displaying the
"more topics" feature in new organizations and streams, where we might
know that all messages in the stream are already available in the
browser.

Based on original work by Roman Godov, and significantly modified by
tabbott.

The second migration involved here could be expensive on Zulip Cloud,
but is unlikely to be an issue on other servers.
2019-03-11 13:30:49 -07:00
Vishnu Ks ef3fbfeb98 events: Include realm_plan_type in initial state data. 2019-03-04 18:46:13 -08:00
Vishnu Ks cc9f00f53e events: Include realm_upload_quota in initial state data. 2019-03-04 18:46:13 -08:00
Tim Abbott 16123c9a58 realm_logo: Fix synchronization of realm night logo.
The night logo synchronization on the settings page was perfect, but
the actual display logic had a few problems:

* We were including the realm_logo in context_processors, even though
  it is only used in home.py.
* We used different variable names for the templating in navbar.html
  than anywhere else the codebase.

* The behavior that the night logo would default to the day logo if
  only one was uploaded was not correctly implemented for the navbar
  position, either in the synchronization for updates code or the
  logic in the navbar.html templates.
2019-03-02 09:45:50 -08:00
Tim Abbott 62dc6dda49 guests: Block guest user access to default streams.
Guest users will just get an empty list of default streams; we also
hide the "Default streams" organization view from the guest users UI.

This is for consistency with not providing guest users the full list
of streams in an organization.
2019-02-28 16:36:30 -08:00
Tim Abbott 1ce0e8256b zoom: Avoid sending Zoom API secret to other admin clients.
Fixing this involves fixing the backend to handle unchanged field
submissions of the Zoom credentials without trying to re-validate the
credentials (for performance) as well as to fetch the already-sent
secret.
2019-02-28 15:43:42 -08:00
sahil839 7157edf4af settings: Add support for uploading logo for night mode.
This adds a new field named realm_night_logo which is used for
displaying the organization logo when the user is in night mode.

Fixes #11176.
2019-02-18 15:15:57 -08:00
YashRE42 22e3955262 search: Add stop words to page_params.
Add all the stop words to page_params, reading from the
`zulip_english.stop` database, with caching to avoid loading the file
on every page load.

Part of #10592.
2019-02-05 10:23:28 -08:00
Anders Kaseorg f0ecb93515 zerver core: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:41:24 -08:00
Hemanth V. Alluri 73d26c8b28 streams: Render and store the stream description from the backend.
This commit does the following three things:
    1. Update stream model to accomodate rendered description.
    2. Render and save the stream rendered description on update.
    3. Render and save stream descriptions on creation.

Further, the stream's rendered description is also sent whenever the
stream's description is being sent.

This is preparatory work for eliminating the use of the
non-authoritative marked.js markdown parser for stream descriptions.
2019-02-01 22:24:18 -08:00
Steve Howell 0ef5d1f9c8 user status: Add "status_text" support in the backend.
You can now pass in an info field with a value
like "out to lunch" to the /users/me/status,
and the server will include that in its outbound
events.

The semantics here are that both "away" and
"status_text" have to have defined values in order
to cause changes.  You can omit the keys or
pass in None when values don't change.

The way you clear info is to pass the empty
string.

We also change page_params to have a dictionary
called "user_status" instead of a set of user
ids.  This requires a few small changes on the
frontend.  (We will add "status_text" support in
subsequent commits; the changes here just keep
the "away" feature working correctly.)
2019-01-24 09:46:35 -08:00
Steve Howell a2614956d5 status: Add away_user_ids to page_params.
(Also, any client that registers for 'user_status'
will see `away_user_ids`.)
2019-01-02 09:16:31 -08:00
Hemanth V. Alluri e3aed0f7bc custom profile fields: Markdown rendering for custom profile field values.
This makes it possible it include our standard markdown formatting in
one's custom profile fields, allowing for links, emphasis, emoji, etc.

Fixes #10131.
2019-01-01 21:06:21 -08:00
Hemanth V. Alluri 28d344b4b5 custom profile fields: Pass value as part of a dictionary.
While we're at it, we remove the JSON parsing that was part of the
user field code path, since this function isn't responsible for
rendering user fields.
2019-01-01 21:05:28 -08:00
Joshua Pan ad1df0ebeb settings: Add support for customizing the top-left logo.
This adds a new realm_logo field, which is a horizontal-format logo to
be displayed in the top-left corner of the webapp, and any other
places where we might want a wide-format branding of the organization.

Tweaked significantly by tabbott to rebase, fix styling, etc.

Fixing the styling of this feature's loading indicator caused me to
notice the loading indicator for the realm_icon feature was also ugly,
so I fixed that too.

Fixes #7995.
2018-12-18 12:44:52 -08:00
Marco Burstein ba46dc83c6 notifications: Add a setting for changing the notification sound.
Also, add a new notification sound, "ding". It comes from
https://freesound.org, where the original Zulip notification sound comes
from as well. In the future, new sounds can be added by adding audio
files to the `static/audio/notification_sounds` directory.

Tweaked significantly by tabbott:
* Avoided removing static/audio/zulip.ogg, because that file is
  checked for by old versions of the desktop app.
* Added a views check for the sound being valid + tests.
* Added additional tests.
* Restructured the test_events test to be cleaner.
* Removed check_bool_or_string.
* Increased max length of notification_sound.
* Provide available_notification_sounds in events data set if global
  notifications settings are requested.

Fixes #8051.
2018-12-09 21:25:30 -08:00
Joshua Pan 441dca0eaf email: Support delivery email in email change system. 2018-12-06 17:43:26 -08:00
Steve Howell 881e046c37 minor: Use TOPIC_NAME constant for "subject" event. 2018-11-08 14:01:45 +00:00
Pragati Agrawal d5df0377cc settings_users: Support guest user in admin-user-table.
This supports guest user in the user-info-form-modal as well as in the
role section of the admin-user-table.

With some fixes by Tim Abbott and Shubham Dhama.
2018-10-29 12:33:35 -07:00
Joshua Pan 83d8d662dc events: Query starred message ids in fetch_initial_state_data. 2018-08-21 13:01:41 -07:00
Yashashvi Dave 6e65235a6d zerver/lib/events.py: Add FIELD_TYPE_CHOICES_DICT to page_params.
This commit add FIELD_TYPE_CHOICES_DICT to page_params and replace
FIELD_TYPE_CHOICES.

FIELD_TYPE_CHOICES_DICT includes all field types with keyword, id
and display name. Using this field-type-dict, we can access field
type information by it's keyword, and remove all static use of
field-type'a name or id in frontend.
This commit also modifies functions in js where this page_params
field-types is used.
2018-08-21 11:37:51 -07:00
Tim Abbott 0068a5ccd6 events: Fix can_subscribe_other_users not being set properly.
I don't understand why this didn't cause test failures in CI; this
change was clearly required and test_change_realm_property was failing
consistently for me locally.
2018-08-21 11:20:59 -07:00
Shubham Padia e21e8c1bae compose: Hide subscribe button and change text for waiting period users.
Fixes #10124.
Users in the waiting period category cannot subscribe other users to
a stream. When a user tries to mention another unsubscribed user, a
warning message appears with a subscribe button on it to subscribe
the other user.
This commit removes the subscribe button and changes the warning text
for users in the waiting period category.
2018-08-13 10:18:35 -07:00
Tim Abbott 58ee3fa8c4 page_params: Include avatars and similar data in cross-realm bots.
This ensures that the format of this data structures matches that for
in-realm bots in the main users data structure (including avatars,
etc.).

Fixes #10138.
2018-08-01 15:09:11 -07:00
Roman Godov 5e70577f84 models: Rename Realm.show_digest_email field.
This renames Realm.show_digest_email field to
digest_emails_enabled, for greater clarity as to what it does
just from seeing the setting name, without having to look it up.

Fixes part of #10042.
2018-08-01 11:05:58 -07:00
Rishi Gupta 0573c17819 stream settings: Set weekly traffic to None instead of -1 for new streams.
The only place this gets used seems to be subscription.handlebars, where a
value of None (or null) is probably less broken-seeming than -1 anyway.
2018-07-23 16:43:15 -07:00
Joshua Pan a9d9919648 Add delivery_email field to page_params.
This allows the frontend to use the `delivery_email`
field for display use.
2018-07-23 15:38:41 -07:00
Aastha Gupta dfde4fac85 invitations: Send 'invites_changed' event for invitations events.
Fixes #7665

In case of invitation events, 'invites_changed' event without
any real payload is sent to all the realm admins and the user.
The event is handled by reloading the list to view recent changes.

Commit tweaked by shubhamdhama:
* Send an `invite_changed` event when an user accept an invite.
    Also, added the test for the same.
* No need to delete the invite list in frontend, current logic
    handles the case when the invite data is changed properly.
* Extracted the common logic for sending an event into
    `notify_invites_changed`.
2018-07-13 18:00:06 +05:30
Yashashvi Dave 649fccde6b zerver/lib/events.py: Fix custom field values not updated in apply_events.
Custom profile field value are stored in different structure compare to
other profile fields in events, so generic way to update fields wasn't
updating custom profile fields in `apply_event` function.

Fix this by adding check for custom fields in `apply_event`.

This also adds the appropriate test_events test to verify this code path.

Fixes part of #9875.
2018-07-10 10:30:23 +05:30
Shubham Dhama 596ce5b60f invite: Make inviting new users inaccessible for guest users. 2018-06-16 06:33:13 -07:00
jkiely a1b5e7d6d5 mypy: Enable strict optional in lib/events.
Modify one variable assignment to pass under strict conditions.
2018-05-17 11:11:53 -07:00
Vishnu Ks 372e9740ac events: Add date_joined to user_dict. 2018-05-17 07:49:35 -07:00
Steve Howell de47eeb6f1 Add /submessage endpoint. 2018-05-16 15:13:33 -07:00
Shubham Padia 295fcb8536 models: Add `is_announcement_only` to stream_dict in `actions.py`.
Adds `is_announcement_only` to `stream_dict`s in order to access
the property in the frontend.
2018-05-13 09:11:51 -07:00
Aditya Bansal a68376e2ba zerver/lib: Change use of typing.Text to str. 2018-05-12 15:22:39 -07:00
Tim Abbott f24630fd4a events: Include data for whether push notifications are enabled.
This is primarily useful for the mobile app, but could also be used to
control whether we display push-notifications related settings to
users in the web UI.
2018-05-08 11:45:13 -07:00
Tim Abbott 6317e4d63c events: Extract post_process_state.
This is a preparatory refactor for potentially being able to call this
from the web-public archive feature's code path.
2018-05-06 23:21:29 -07:00
Tim Abbott 112805251f fetch_initial_state_data: Use realm local variable.
This is just a simple code cleanup.
2018-05-06 23:20:52 -07:00
Tim Abbott 956bd74905 attachments: Send events for attachment updates.
We send add events on upload, update events when sending a message
referencing it, and delete updates on removal.

This should make it possible to do real-time sync for the attachments
UI.

Based in part on work by Aastha Gupta.
2018-05-04 16:22:27 -07:00
Tim Abbott 69c4645bd2 attachments: Stop fetching attachments in / endpoint.
We only use this data in a rarely-used settings screen, and it can be
large after years of posting screenshots.

So optimize the performance of / by just loading these data when we
actually visit the page.

This saves about 300ms of runtime for loading the home view for my
user account on chat.zulip.org.
2018-05-04 16:22:26 -07:00
Tim Abbott fdc1182a76 events: Optimize query for custom profile fields.
Our query for Custom Profile fields was for no good reason passing the
list of all users in the realm (potentially many thousands) into a
database query, rather than letting the database do that join.

Fixing this saves 100ms-200ms in the loading time for / on
chat.zulip.org for all users, since we were previously doing a ton of
work even if the feature wasn't being used.
2018-05-04 12:59:30 -07:00
Yashashvi Dave 66759358e2 create stream: Add maxlength restriction on name and description. 2018-04-30 10:11:25 -07:00
Tim Abbott 7d6bb3dcb4 settings: Remove obsolete default_desktop_notifications setting.
This actually hasn't been hooked up to do anything in years.

While we're at it, we remove the entire "Zulip Labs" settings page.
2018-04-28 13:46:07 -07:00
YJDave 6bef44a9fa org setting: Add time limits for message deleting.
Add realm setting to set time limit for message deleitng.
Set default value of message_content_delete_limit_seconds
to 600 seconds(10 min).

Thanks to Shubham Dhama for rebasing and reworking this.  Some final
edits also done by Tim Abbott.

Fixes #7344.
2018-04-27 19:22:28 -07:00
Vishnu Ks c9e932a7ce settings: Add support for Hangouts as the video chat provider.
The only thing that's annoying about this feature is that you need to
be a paying G Suite customer to use it.
2018-04-23 09:39:47 -07:00
Umair Khan 7885dd4408 profile: Send field types to client side. 2018-03-29 13:09:12 -07:00
Greg Price b42a7b1701 digest: Add a server setting, and disable by default.
This feature isn't really ready yet -- the relevance isn't good, so
the emails aren't a great experience.  More work needed; pending that,
just don't send them.

There's already a per-realm setting, which doesn't have a control in
the org settings UI but does suppress it in the per-user settings UI.
Piggyback on that to suppress that UI control when the feature is
disabled at the server level too.

Also cut a comment that hasn't really made sense since the logic was
changed months ago -- the comment originally explained why we sent
digests on Tuesday, Wednesday, and Thursday, and doesn't correspond to
why we dialled back to weekly on Tuesdays.
2018-03-23 14:12:01 -07:00
Sarah f5c2fb8438 org settings: Create backend api for allow_community_topic_editing.
Adds the code for updating the allow_community_topic_editing
setting.
2018-03-22 16:02:27 -07:00
Tim Abbott 25d9731a3c custom profiles: Fix totally broken events.py logic.
Apparently, my manual testing here was in error; the new version
actaully works for delivering custom profile data to the frontend.
2018-03-11 21:25:23 -07:00
Tim Abbott 02b8453367 custom profiles: Send custom profile data to frontend.
This will fetch the data of custom fields for all users.
2018-03-11 18:08:17 -07:00
Shubham Padia 32e38d36aa bots: Send add/delete event on bot ownership change.
Adds realm_bot delete event. On bot ownership change, add event is
sent to the bot_owner(if not admin) and delete event to the
previous bot owner(if not admin). For admin, update event is sent.
2018-03-08 07:54:19 -08:00
Archana BS c7650c0d9d streams: send data for recent streams traffic to frontend.
Significantly edited by Tim Abbott and Vishnu Ks.
2018-03-04 13:24:53 -08:00
Robert Hönig 312cabb9a6 models.py: Move two long functions with circular imports to actions.py.
models.py should only contain thin wrapper functions. Furthermore,
this move allows us to remove the circular imports. The two moved
functions are interdependent and are thus moved in one commit.
2018-03-01 08:25:43 -08:00
Tim Abbott b84f2223a5 avatars: Always include a working URL for the user's own avatar.
This fixes an issue where the user's own avatar was being sent down
the wire as None.  We could have fixed it, as in #8265, by adding code
in the webapp and mobile apps to compute medium-size gravatar URLs as
well, but that would be messy, and there's little benefit to that
complexity (saving at most 2 URLs from the payload).

Fixes #8253.
2018-02-05 12:45:41 -08:00
Greg Price 55cf54c087 upload: Remove old per-user quota feature.
We'll replace this primarily with per-realm quotas (plus the simple
per-file limit of settings.MAX_FILE_UPLOAD_SIZE, 25 MiB by default).

We do want per-user quotas too, but they'll need some more management
apparatus around them so an admin has a practical way to set them
differently for different users.  And the error handling in this
existing code is rather confused.  Just clear this feature out
entirely for now; then we'll build the per-realm version more cleanly,
and then we can later add back per-realm quotas modelled after that.

The migration to actually remove the field is in a subsequent commit.

Based in part on work by Vishnu Ks (hackerkid).
2018-01-29 16:06:11 -08:00
Robert Hönig d1d8365a6b embedded bots: Add config data UI.
This adds UI fields in the bot settings for specifying
configuration values like API keys for a bot. The names
and placeholder values for each bot's config fields are
fetched from the bot's <bot>.conf template file in the
zulip_bots package. This also adds giphy and followup
as embedded bots.
2018-01-09 11:50:54 -05:00
YJDave fa44d2ea69 settings: Remove autoscroll_forever setting.
Fixes #6845
2018-01-02 10:35:49 -05:00
Marco Burstein 9327048f6f mypy: Use Python 3 type syntax in `zerver/lib/events.py`. 2017-12-22 08:17:00 -05:00
rht 3f4bf2d22f zerver/lib: Use python 3 syntax for typing.
Extracted from a larger commit by tabbott because these changes will
not create significant merge conflicts.
2017-11-21 20:56:40 -08:00
Vishnu Ks 34689370cd settings: Create UI and backend for setting signups stream. 2017-11-21 17:39:51 -08:00
Tim Abbott 054952a44a docs: Update links from codebase to point to ReadTheDocs. 2017-11-16 10:53:49 -08:00
Umair Khan 98be0cc502 user-groups: Send delete user group event.
Fixes #7380
2017-11-15 22:35:43 -08:00
Umair Khan 79f4a7627f user-groups: Send remove members event. 2017-11-15 22:35:23 -08:00
Umair Khan cec3f19366 user-groups: Send add members event. 2017-11-15 22:35:23 -08:00
Umair Khan 5571122120 user-groups: Send name update event. 2017-11-15 22:35:23 -08:00
Umair Khan 912505317a user-groups: Send create group event. 2017-11-15 22:35:23 -08:00
rht 09af29b051 zerver/lib: Text-wrap long lines exceeding 110. 2017-11-15 10:58:03 -08:00
Vishnu Ks f44b60a150 Implement API for default stream groups. 2017-11-10 16:05:36 -08:00
Umair Khan 4c6a376fd5 user-groups: Add groups to page_params. 2017-11-09 17:35:46 -08:00
Steve Howell 8616a4ebc5 Use client_gratavar=True for users on in page_params.
This change affects realm_users and realm_non_active_users.

Note that we still send full avatar urls in realm_user/add
events, so apply_events has to do something mildly hacky to
turn the avatar_url to None in that case.

Fixing the event is probably not worth the trouble, as single
urls are not bandwidth hogs; we only need this optimization
for bulk data.
2017-11-07 16:38:14 -08:00
Steve Howell a076581426 Use client_gravatar=True for avatar URLs in page_params.
This change affects these values:

    * page_params.avatar_url
    * page_params.avatar_url_medium

It requires passing the client_gravatar flag through this
codepath:

    * home_real
    * do_events_register
    * fetch_initial_state_data
    * avatar_url
2017-11-07 16:38:14 -08:00
rht 6cce0e346e refactor: Remove six.moves.filter import. 2017-11-07 10:51:44 -08:00
rht 80a8d4f9f3 refactor: Remove six.moves.map import. 2017-11-07 10:46:42 -08:00
Steve Howell 2bbfda041a Support client_gravatar field for event registration.
This commit allows clients to register client_gravatar=True, and
then we recognize that flag for message events.  If the flag is
True, we will not calculate gravatar URLs and let the clients do
it themselves.  (Clients can calculate gravatar URLs based on
emails with just a little bit of code.)
2017-11-07 10:42:17 -08:00
Tim Abbott 8e2cdedf9a lint: Fix lines in Python codebase longer than 120 characters. 2017-10-26 17:47:30 -07:00
Steve Howell 305fe6106b Add `realm_non_active_users` data to initial payload.
We now add `realm_non_active_users` to the result of
`do_events_register` (and thus `page_params`).  It has
the same structure as `realm_users`, but it's for
non-active users.  Clients need data on non-active users
when they process old messages that were sent by those
users when they were active.  Clients can currently get
most of the data they need in the message events, but it
makes for ugly client code.

Fixes #4322
2017-10-25 11:18:30 -07:00
Steve Howell b94c062368 Make the realm user cache include non-active users.
This is a prepatory commit that adds non-active users to
the realm user cache.  It mostly involves name changes and
removing an `is_active` filter from the relevant DB query.

The only consumer of this cache is `get_raw_user_data`, which
now filters on `is_active` in a dictionary comprehension (but
this will get moved around a bit in a subsequent commit).
2017-10-25 11:18:30 -07:00
Steve Howell 769c741c7c refactor: Simplify event updates for `realm_users`.
We make a few things cleaner for populating `realm_users`
in `do_event_register` and `apply_events`:

    * We have a `raw_users` intermediate dictionary that
      makes event updates O(1) and cleaner to read.

    * We extract an `is_me` section for all updates that
      apply to the current user.

    * For `update` events, we do a more surgical copying
      of fields from the event into our dict.  This
      prevents us from mutating fields in the event,
      which was sketchy (at least in test mode).  In
      particular, this allowed us to remove some ugly
      `del` code related to avatars.

    * We introduce local vars `was_admin` and `now_admin`.

The cleanup had two test implications:

    * We no longer need to normalize `realm_users`, since
      `apply_events` now sees `raw_users` instead.  Since
      `raw_users` is a dict, there is no need to normalize
      it, unlike lists with possibly random order.

    * We updated the schema for avatar updates to include
      the two fields that we used to hackily delete from
      an event.
2017-10-25 11:18:30 -07:00
Tim Abbott 3ee53d5de3 auth: Don't offer password reset links when useless.
If an organization doesn't have the EmailAuthBackend (which allows
password auth) enabled, then our password reset form doesn't do
anything, so we should hide it in the UI.
2017-10-24 12:07:43 -07:00
Steve Howell a66409de2e tests: Add FetchQueriesTest.test_queries().
This new test solves the problem that when we
made changes to the page-load codepath in the past,
it's been hard to identify what new code caused
more database queries.  Now you can see query
counts broken out by event type.

This requires a small, harmless change to extract
an `always_want` function in `lib/events.py`.
2017-10-23 11:11:19 -07:00
derAnfaenger 07703daf90 page_params: Add list of available embedded bots. 2017-10-19 10:37:55 -07:00
Steve Howell fed972d1fb Fix bug with applying message events to unread counts.
The `is_mentioned` flag in message events was buggy.  We now
look directly at flags.

We will kill off `is_mentioned` in a subsequent commit.

We also remove some debugging code in the test that was failing
before this fix.  The test would only fail when `is_mentioned`
was wrong, which never happened when you ran a single test, and
which would happen randomly when you ran multiple tests.
2017-10-11 16:55:34 -07:00
Steve Howell 3e6bfe1b23 Use user_ids, not emails, for bulk stream operations.
We now return user_ids for subscribers to streams in add-stream
events.  This allows us to eliminate the UserLite class for
both bulk adds and bulk removes.  It also simplifies some JS
code that already wanted to use user_ids, not emails.

Fixes #6898
2017-10-08 12:31:12 -07:00
Steve Howell e56084fcf7 Simplify how we apply events for unread messages.
The logic to apply events to page_params['unread_msgs'] was
complicated due to the aggregated data structures that we pass
down to the client.

Now we defer the aggregation logic until after we apply the
events.  This leads to some simplifications in that codepath,
as well as some performance enhancements.

The intermediate data structure has sets and dictionaries that
generally are keyed by message_id, so most message-related
updates are O(1) in nature.

Also, by waiting to compute the counts until the end, it's a
bit less messy to try to keep track of increments/decrements.
Instead, we just update the dictionaries and sets during the
event-apply phase.

This change also fixes some corner cases:

    * We now respect mutes when updating counts.
    * For message updates, instead of bluntly updating
      the whole topic bucket, we update individual
      message ids.

Unfortunately, this change doesn't seem to address the pesky
test that fails sporadically on Travis, related to mention
updates.  It will change the symptom, slightly, though.
2017-10-05 09:42:20 -07:00
rht 035ed93111 zerver/lib: remove `import six`. 2017-09-27 19:10:28 -07:00
rht 2e12fe5e2e zerver/lib: Remove print_function. 2017-09-27 18:05:45 -07:00
rht f43e54d352 zerver/lib: Remove absolute_import. 2017-09-27 10:00:39 -07:00
Steve Howell 428d3027c2 Only require ids for finding DefaultStream objects.
We don't need full Realm objects to find DefaultStream
objects for a realm.  So now a few functions related to
adding/removing default streams use realm_id for lookups.

Similarly, we don't need a full Stream object to find
out if a stream exists in DefaultStream, so we do id
lookups there as well.

This sets us up to use thinner objects in callers.
2017-09-20 10:31:33 -07:00