Commit Graph

9505 Commits

Author SHA1 Message Date
Tim Abbott 51eeb0a3ee cache: Add missing : in test-backend key prefixes.
Previously, these cache keys looked like:
:1:9c26164d3a393e316e0f8210efe270e08710d45astream_by_realm_and_name:...

Now, they look like this:
:1:9c26164d3a393e316e0f8210efe270e08710d45a:stream_by_realm_and_name:...
2019-03-18 10:56:50 -07:00
Tim Abbott 5d20138d72 context_processors: Optimize calculation of social_backends.
This avoids a bunch of duplicated calls to auth_enabled_helper for our
social auth backends, which added up because auth_enabled_helper can
take 100us to run.
2019-03-17 15:20:36 -07:00
Tim Abbott a9350caee2 context_processors: Optimize calculation of no_auth_enabled.
This saves an unnecessary call to auth_enabled_helper.
2019-03-17 15:19:06 -07:00
Tim Abbott cb78ddc491 models: Fix performance of supported_auth_backends with caching.
See the comment, but this is a significant performance optimization
for all of our pages using common_context, because this code path is
called more than a dozen times (recursively) by common_context.
2019-03-17 15:14:05 -07:00
Tim Abbott 79ae9175bb context_processors: Avoid useless duplicate queries for realm object.
We have a few code paths that call get_realm_from_request multiple
times on the same request (e.g. the login page), once inside the view
function and once inside the common context processor code.  This
change saves a useless duplicate database query in those code paths.
2019-03-17 14:08:53 -07:00
Tim Abbott f270e354b1 login: Fix unnecessary queries on /login/ in development.
This block of code with 2 database queries is solely for the /devlogin
endpoint.  Removing that block from the /login code path makes it
easier to test /login perf in development.
2019-03-17 14:03:57 -07:00
Tim Abbott 0a7b6909b9 login: Fix a useless query with DevAuthBackend.
This block was setting `realm` to a value that it already was if you
read a bit further up in the function.
2019-03-17 14:03:27 -07:00
Hemanth V. Alluri 1269b72f47 context_processors: Do not render inline previews for realm description.
We never intended to render them for this use case as the result would
not look good, and now we have a convenient bugdown option for
controlling this behavior.

Since we're not storing the markdown rendering anywhere, there's
conveniently no data migration required.

Fixes #11889.
2019-03-17 13:55:10 -07:00
Mateusz Mandera a269c4bdd4 email_mirror: Make email_token a unique column of Stream. 2019-03-17 12:55:35 -07:00
Eeshan Garg 623ee15bee webhooks/bitbucket: Account for missing user info.
According to our logs, some BitBucket enterprise payloads may
not contain the name of the user who pushed all the commits.
2019-03-16 15:09:03 -02:30
sameerchoubey a47edd4fc8 webhooks/jira: Use dictionary to map events to functions. 2019-03-16 14:27:00 -02:30
Ben Muschol d526ff00f2 settings: Rename "user avatar" to "profile picture"
This renames references to user avatars, bot avatars, or organization
icons to profile pictures. The string in the UI are updated,
in addition to the help files, comments, and documentation. Actual
variable/function names, changelog entries, routes, and s3 buckets are
left as-is in order to avoid introducing bugs.

Fixes #11824.
2019-03-15 13:29:56 -07:00
Tim Abbott 0a145f2203 test_email_notifications: Fix a potentially flaky test.
The previous code assumed the `zulip` realm had ID 1.
2019-03-15 11:02:17 -07:00
Tim Abbott c7ebfdc978 notifications: Rename test_notifications to test_email_notifications.
Similar to the last commit, this makes the scope of this test suite
clearer.
2019-03-15 11:02:17 -07:00
Tim Abbott 50dc317466 notifications: Rename notifications.py to email_notifications.py.
This library is entirely about email notifications specifically, and
this rename should help make the codebase more readable.
2019-03-15 11:02:17 -07:00
Tim Abbott 28a4c143c8 emails: Move clear_scheduled_*emails to send_email.py.
It more properly belongs as part of our outgoing email library, not
notifications.py (which is really about email notifications).
2019-03-15 11:02:17 -07:00
Raymond Akornor 89351cdd19 send_email: Add ScheduledEmail support for multiple recipients.
Follow up on 92dc363. This modifies the ScheduledEmail model
and send_future_email to properly support multiple recipients.

Tweaked by tabbott to add some useful explanatory comments and fix
issues with the migration.
2019-03-15 11:02:12 -07:00
Tim Abbott 4f02795830 home: Fix parsing bug for initial narrows.
There's more we'll want to do here, but this at least avoids error
emails when users attempt to misuse this interface.
2019-03-14 14:53:03 -07:00
Tim Abbott 983e24a7f5 auth: Use HTTP status 404 for invalid realms.
Apparently, our invalid realm error page had HTTP status 200, which
could be confusing and in particular broken our mobile app's error
handling for this case.
2019-03-14 13:50:09 -07:00
Puneeth Chaganti d75d2c9974 soft-deactivation: Run catch-up when "auto" deactivate is run.
When soft deactivation is run for in "auto" mode (no emails are
specified and all users inactive for specified number of days are
deactivated), catch-up is also run in the "auto" mode if
AUTO_CATCH_UP_SOFT_DEACTIVATED_USERS is True.

Automatically catching up soft-deactivated users periodically would
ensure a good user experience for returning users, but on some servers
we may want to turn off this option to save on some disk space.

Fixes #8858, at least for the default configuration, by eliminating
the situation where there are a very large number of messages to recover.
2019-03-14 11:53:15 -07:00
Puneeth Chaganti c61d3420e8 soft-deactivation: Extract getting users from emails to a function. 2019-03-14 20:57:00 +05:30
Puneeth Chaganti f5c61e7d31 soft-deactivation: Patch logging.info to clean up test output. 2019-03-14 09:26:25 +05:30
Puneeth Chaganti ff07ca7417 soft-deactivation: Slightly improve the test for catching up users. 2019-03-14 09:26:06 +05:30
Puneeth Chaganti cf65136002 soft-deactivation: Add code to catch up soft deactivated users. 2019-03-13 17:23:14 -07:00
Puneeth Chaganti 52afbe5e8d soft-deactivation: Rename maybe_catch_up_soft_deactivated_user.
Rename `maybe_catch_up_soft_deactivated_user` to
`reactivate_user_if_soft_deactivated`.
2019-03-13 17:16:22 -07:00
Puneeth Chaganti 7f06f6bd73 soft-deactivation: Raise exception with all missing emails at once. 2019-03-13 17:16:22 -07:00
Puneeth Chaganti 82d9789d93 soft-deactivation: Paginate bulk creation of UserMessage rows.
A user who has been soft deactivated for a long time might have 10Ks of message
history that was "soft deactivated". It might take a minute or more to add
UserMessage rows for all of these messages, causing timeouts. So, we paginate
the creation of these UserMessage rows.
2019-03-13 17:16:22 -07:00
Puneeth Chaganti 8c5a0f585b soft-deactivation: Clarify that value being fetched is recipient_id. 2019-03-13 17:16:22 -07:00
Puneeth Chaganti 4d77ffe2cb soft-deactivation: Refactor fetching list of existing UserMessages. 2019-03-13 17:16:22 -07:00
Puneeth Chaganti 8c5425e33a soft-deactivation: Remove unnecessary select_related calls.
It will become more clear in the upcoming commits, but these calls
aren't actually useful as we're only extracting a few IDs.
2019-03-13 17:15:50 -07:00
Harshit Bansal 262eb42b77 auth: Reverse the `sort_order` parameter's semantics.
This will make sure that if a backend doesn't specify a values for
`sort_order` parameter then it will sorted to the bottom not at the
top.
2019-03-13 14:44:57 -07:00
Tim Abbott 412d35900f slack import: Fix handling of tombstone files.
Apparently, the mode attribute is not always present.
2019-03-13 14:39:20 -07:00
Tim Abbott 76a06c8e54 send_test_email: Clean up output and provide advice.
Previously, while we sent emails using both noreply addresses, we
didn't make clear what was going on, leading to some potential
confusion.
2019-03-13 14:27:52 -07:00
Tim Abbott 662fa037f2 events: Don't assume that streams/subscriptions were requested.
This fixes a bug in our race condition handling in the case that
streams and/or subscriptions register content was not requested by the
client.
2019-03-13 14:12:59 -07:00
Tim Abbott 4694de31c5 tests: Fix linter error in test_management_commands. 2019-03-13 14:06:46 -07:00
Tim Abbott 49680a4503 slack import: Skip processing tombstone files.
The tombstone files undocumented feature of Slack's export format
appears sometimes and has no real data, so we just need to skip these.

Fixes #11619.
2019-03-13 12:43:11 -07:00
Hemanth V. Alluri 99c3e2ecdc management: Support sending custom headers when testing a webhook.
this commit adds an option to specify custom headers when using the
`./manage.py send_webhook_fixture_message` tool.
2019-03-13 12:40:47 -07:00
vsvipul e859ab7545 portico: Fix broken electron check condition for password reset.
This logic for passing through whether the user was logged in never
worked, because we were trying to read the client.

Fix this, and add tests to ensure it never breaks again.

Restructured by tabbott to have completely different code with the
same intent.

Fixes #11802.
2019-03-12 11:25:29 -07:00
Harshit Bansal a6e523f9e4 ldap: Ensure email is valid for realm before registering.
Previously, the LDAP authentication model ignored the realm-level
settings for who can join a realm.  This was sort of reasonable at the
time, because the original LDAP auth was an SSO solution that didn't
allow multiple realms, and so one could fully configure authentication
settings on the LDAP side.  But now that we allow multiple realms with
the LDAP backend, one could easily imagine wanting different
restrictions on them, and so it makes sense to add this enforcement.
2019-03-12 11:09:18 -07:00
Challa Venkata Raghava Reddy b69aec2dbc streams: Add first_message_id tracking first message in stream.
This field is primarily intended to support avoiding displaying the
"more topics" feature in new organizations and streams, where we might
know that all messages in the stream are already available in the
browser.

Based on original work by Roman Godov, and significantly modified by
tabbott.

The second migration involved here could be expensive on Zulip Cloud,
but is unlikely to be an issue on other servers.
2019-03-11 13:30:49 -07:00
Vishnu Ks 9671ed7bab subdomains: Extract get_subdomain_from_hostname function. 2019-03-11 12:01:11 -07:00
Rohitt Vashishtha 51986c27a5 markdown: Use common testcases for realmfilters in topics and messages.
The actual bug in #11791 was caused by code reverted in
3ed85f4cd7, so technically #11791 is
already fixed.  However, it makes sense to add tests to ensure that it
doesn't regress in the future as part of closing out the issue.

Fixes #11791.
2019-03-11 11:48:47 -07:00
Tim Abbott c05fb01cbf stream: Fix validator for stream colors.
Apparently, our new validator for stream color having a valid format
incorrectly handled colors that had duplicate characters in them.

(This is caused in part by the spectrum.js logic automatically
converting #ffff00 to #ff0, which our validator rejected).  Given that
we had old stream colors in the #ff0 format in our database anyway for
legacy, there's no benefit to banning these colors.

In the future, we could imagine standardizing the format, but doing so
will require also changing the frontend to submit colors only in the
6-character format.

Fixes an issue reported in
https://github.com/zulip/zulip/issues/11845#issuecomment-471417073
2019-03-11 11:43:39 -07:00
Eeshan Garg 222966b6ba webhooks/zapier: Support authentication for the Zapier Zulip app.
If the user sets up a Zap using an incoming webhook bot's API
key, the authentication goes through our webhook.
2019-03-11 10:54:55 -07:00
Eeshan Garg 7b6a37780a webhooks/github: Ignore organization and milestone events.
These events are not super useful and were cluttering up our
webhook logs.
2019-03-10 14:13:17 -07:00
Eeshan Garg 6afd02bef5 webhooks/github: Restrict membership event scope to teams.
According to GitHub's webhook docs, the scope of a membership
event can only be limited to 'teams', which holds true when a
new member is added to a team. However, we just found a payload
in our logs that indicates that when a user is removed from a
team, the scope of the membership is erroneously set to
'organization', not 'team'. This is most likely a bug on
GitHub's end because such behaviour is a direct violation of
their webhook API event specifications. We account for this
by restricting membership events to teams explicitly, at least
till GitHub's docs suggest otherwise.
2019-03-10 14:13:17 -07:00
Puneeth Chaganti abc9a7a4f9 digest: Use a list of Message objects to gather_hot_conversations.
Instead of a queryset of UserMessage objects, `gather_hot_conversations` now
accepts a list of Message objects.
2019-03-09 23:25:26 -08:00
Puneeth Chaganti cb5e9107f4 digest: Directly fetch recipient ids from the DB.
Instead of iterating over Subscriptions and creating the list of home view
recipients, the query now only fetches recipient IDs from the DB.
2019-03-09 23:25:26 -08:00
Harshit Bansal 94649f58f2 tests: Refactor `query_ldap()` and add complete test coverage. 2019-03-09 22:12:51 -08:00
Harshit Bansal b519e6594e management: Move `query_ldap` function to `zproject/backends.py`.
This will make it simpler to organize and unit-test all of our
authentication backend code.
2019-03-09 22:12:36 -08:00