Commit Graph

31159 Commits

Author SHA1 Message Date
Puneeth Chaganti d75d2c9974 soft-deactivation: Run catch-up when "auto" deactivate is run.
When soft deactivation is run for in "auto" mode (no emails are
specified and all users inactive for specified number of days are
deactivated), catch-up is also run in the "auto" mode if
AUTO_CATCH_UP_SOFT_DEACTIVATED_USERS is True.

Automatically catching up soft-deactivated users periodically would
ensure a good user experience for returning users, but on some servers
we may want to turn off this option to save on some disk space.

Fixes #8858, at least for the default configuration, by eliminating
the situation where there are a very large number of messages to recover.
2019-03-14 11:53:15 -07:00
Puneeth Chaganti c61d3420e8 soft-deactivation: Extract getting users from emails to a function. 2019-03-14 20:57:00 +05:30
Puneeth Chaganti f5c61e7d31 soft-deactivation: Patch logging.info to clean up test output. 2019-03-14 09:26:25 +05:30
Puneeth Chaganti ff07ca7417 soft-deactivation: Slightly improve the test for catching up users. 2019-03-14 09:26:06 +05:30
Vaibhav d285f67afe copy_and_paste: Fix copy/paste behaviour on Firefox.
Firefox and Chrome handle selection of multiple messages differently. FF
creates multiple Ranges and Chrome creates one. Code written previously
terminated when we got an empty Range or Range with same starting and ending
message. This behaviour was incorrect since the selection was valid.

* Check for same message is done after looping through all the ranges now.
* `construct_copy_div` is called once since it is much easier to determine
  start_id and end_id with confidence and this avoids any overlap between
  same message ids.

Extended by tabbott to include a ton of comments on how this works.

Fixes #11805.
2019-03-13 17:37:26 -07:00
Puneeth Chaganti cf65136002 soft-deactivation: Add code to catch up soft deactivated users. 2019-03-13 17:23:14 -07:00
Puneeth Chaganti 52afbe5e8d soft-deactivation: Rename maybe_catch_up_soft_deactivated_user.
Rename `maybe_catch_up_soft_deactivated_user` to
`reactivate_user_if_soft_deactivated`.
2019-03-13 17:16:22 -07:00
Puneeth Chaganti 7f06f6bd73 soft-deactivation: Raise exception with all missing emails at once. 2019-03-13 17:16:22 -07:00
Puneeth Chaganti 82d9789d93 soft-deactivation: Paginate bulk creation of UserMessage rows.
A user who has been soft deactivated for a long time might have 10Ks of message
history that was "soft deactivated". It might take a minute or more to add
UserMessage rows for all of these messages, causing timeouts. So, we paginate
the creation of these UserMessage rows.
2019-03-13 17:16:22 -07:00
Puneeth Chaganti 8c5a0f585b soft-deactivation: Clarify that value being fetched is recipient_id. 2019-03-13 17:16:22 -07:00
Puneeth Chaganti 4d77ffe2cb soft-deactivation: Refactor fetching list of existing UserMessages. 2019-03-13 17:16:22 -07:00
Puneeth Chaganti 8c5425e33a soft-deactivation: Remove unnecessary select_related calls.
It will become more clear in the upcoming commits, but these calls
aren't actually useful as we're only extracting a few IDs.
2019-03-13 17:15:50 -07:00
Harshit Bansal 262eb42b77 auth: Reverse the `sort_order` parameter's semantics.
This will make sure that if a backend doesn't specify a values for
`sort_order` parameter then it will sorted to the bottom not at the
top.
2019-03-13 14:44:57 -07:00
Tim Abbott 412d35900f slack import: Fix handling of tombstone files.
Apparently, the mode attribute is not always present.
2019-03-13 14:39:20 -07:00
Tim Abbott 76a06c8e54 send_test_email: Clean up output and provide advice.
Previously, while we sent emails using both noreply addresses, we
didn't make clear what was going on, leading to some potential
confusion.
2019-03-13 14:27:52 -07:00
Tim Abbott 662fa037f2 events: Don't assume that streams/subscriptions were requested.
This fixes a bug in our race condition handling in the case that
streams and/or subscriptions register content was not requested by the
client.
2019-03-13 14:12:59 -07:00
Tim Abbott cdf987fb14 push-translations: Run manage.py makemessages first.
This saves an extra step in pushing translations to transifex, with no
real negative side effects.
2019-03-13 14:11:27 -07:00
Tim Abbott c5a2bb2c8b i18n: Update translation data from Transifex. 2019-03-13 14:10:09 -07:00
Tim Abbott 4694de31c5 tests: Fix linter error in test_management_commands. 2019-03-13 14:06:46 -07:00
Tim Abbott 49680a4503 slack import: Skip processing tombstone files.
The tombstone files undocumented feature of Slack's export format
appears sometimes and has no real data, so we just need to skip these.

Fixes #11619.
2019-03-13 12:43:11 -07:00
Hemanth V. Alluri 99c3e2ecdc management: Support sending custom headers when testing a webhook.
this commit adds an option to specify custom headers when using the
`./manage.py send_webhook_fixture_message` tool.
2019-03-13 12:40:47 -07:00
vsvipul e859ab7545 portico: Fix broken electron check condition for password reset.
This logic for passing through whether the user was logged in never
worked, because we were trying to read the client.

Fix this, and add tests to ensure it never breaks again.

Restructured by tabbott to have completely different code with the
same intent.

Fixes #11802.
2019-03-12 11:25:29 -07:00
Harshit Bansal a6e523f9e4 ldap: Ensure email is valid for realm before registering.
Previously, the LDAP authentication model ignored the realm-level
settings for who can join a realm.  This was sort of reasonable at the
time, because the original LDAP auth was an SSO solution that didn't
allow multiple realms, and so one could fully configure authentication
settings on the LDAP side.  But now that we allow multiple realms with
the LDAP backend, one could easily imagine wanting different
restrictions on them, and so it makes sense to add this enforcement.
2019-03-12 11:09:18 -07:00
Anders Kaseorg fd6f18f7cf nginx: Improve TLS settings based on Mozilla config generator.
Lengthen the session timeout and enlarge the session cache.  Upgrade
Diffie-Hellman parameters from fixed 1024-bit to custom 2048-bit.
Enable OCSP stapling.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-03-11 23:40:34 -07:00
Siddharth Varshney 0d25baedfa compose: Fix narrow button text when switching to PM.
This changes the "new private message" button to be instead "new
conversation" when looking at PMs, to avoid confusion that the button
was the right thing to do to reply to the current private message
conversation.

Fixes #11679.
2019-03-11 21:58:05 -07:00
Anders Kaseorg aad61c42a3 css: Replace generated U+202A LEFT-TO-RIGHT EMBEDDING with CSS properties.
These generated characters (added in #9889) were causing poor wrapping
behavior, at least in Firefox.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-03-11 17:12:53 -07:00
Steve Howell 597c812300 refactor: Use a Dict to track active topic widgets.
Even though there are only ever zero or one active
topic widgets in our current sidebar, it's almost the
same amount of code to just manage them with a Dict.

Also, we can more easily do possible future features
like setting streams to be always-open.
2019-03-11 15:49:44 -07:00
Challa Venkata Raghava Reddy b69aec2dbc streams: Add first_message_id tracking first message in stream.
This field is primarily intended to support avoiding displaying the
"more topics" feature in new organizations and streams, where we might
know that all messages in the stream are already available in the
browser.

Based on original work by Roman Godov, and significantly modified by
tabbott.

The second migration involved here could be expensive on Zulip Cloud,
but is unlikely to be an issue on other servers.
2019-03-11 13:30:49 -07:00
Vishnu Ks 8eeb8280b4 activity: Create interface for doing support operations.
This should grow into a tool that makes it much easier to do common
organization management tasks without using a manage.py shell.
2019-03-11 12:01:11 -07:00
Vishnu Ks 42de9a0c71 analytics: Extract out the function for getting plan name. 2019-03-11 12:01:11 -07:00
Vishnu Ks ca1276961d billing: Create get_discount_for_realm function. 2019-03-11 12:01:11 -07:00
Vishnu Ks 9671ed7bab subdomains: Extract get_subdomain_from_hostname function. 2019-03-11 12:01:11 -07:00
Vishnu Ks 51dcc5fdde templates: Add django timesince filter to jinja2 filters. 2019-03-11 12:01:11 -07:00
Rohitt Vashishtha 51986c27a5 markdown: Use common testcases for realmfilters in topics and messages.
The actual bug in #11791 was caused by code reverted in
3ed85f4cd7, so technically #11791 is
already fixed.  However, it makes sense to add tests to ensure that it
doesn't regress in the future as part of closing out the issue.

Fixes #11791.
2019-03-11 11:48:47 -07:00
Tim Abbott c05fb01cbf stream: Fix validator for stream colors.
Apparently, our new validator for stream color having a valid format
incorrectly handled colors that had duplicate characters in them.

(This is caused in part by the spectrum.js logic automatically
converting #ffff00 to #ff0, which our validator rejected).  Given that
we had old stream colors in the #ff0 format in our database anyway for
legacy, there's no benefit to banning these colors.

In the future, we could imagine standardizing the format, but doing so
will require also changing the frontend to submit colors only in the
6-character format.

Fixes an issue reported in
https://github.com/zulip/zulip/issues/11845#issuecomment-471417073
2019-03-11 11:43:39 -07:00
Pragati Agrawal e630dde240 settings_org: Improve robustness of organization settings.
This moves the configuration of widget type from settings_org to instead
live in respective HTML templates, via `data-widget-setting-type` and we
also remove `get_subsection_property_types` and refactor function
`populate_data_for_request` accordingly.

Fixes: #11708.
2019-03-11 10:58:15 -07:00
Eeshan Garg 222966b6ba webhooks/zapier: Support authentication for the Zapier Zulip app.
If the user sets up a Zap using an incoming webhook bot's API
key, the authentication goes through our webhook.
2019-03-11 10:54:55 -07:00
Eeshan Garg 7b6a37780a webhooks/github: Ignore organization and milestone events.
These events are not super useful and were cluttering up our
webhook logs.
2019-03-10 14:13:17 -07:00
Eeshan Garg 6afd02bef5 webhooks/github: Restrict membership event scope to teams.
According to GitHub's webhook docs, the scope of a membership
event can only be limited to 'teams', which holds true when a
new member is added to a team. However, we just found a payload
in our logs that indicates that when a user is removed from a
team, the scope of the membership is erroneously set to
'organization', not 'team'. This is most likely a bug on
GitHub's end because such behaviour is a direct violation of
their webhook API event specifications. We account for this
by restricting membership events to teams explicitly, at least
till GitHub's docs suggest otherwise.
2019-03-10 14:13:17 -07:00
Tim Abbott 0a17a2acaa docs: Recommend using an SSD for the Zulip database.
This is sorta common knowledge for folks who have managed databases,
but not everyone has.
2019-03-10 13:51:52 -07:00
Pragati Agrawal 9eed78433e node_tests: Refactor `test_change_save_button_state` in settings_org.
This refactors `test_change_save_button_state` function using ES6 syntax,
to make it more clean and readable.
2019-03-10 13:09:36 -07:00
Pragati Agrawal 36fcb67812 settings_org: Fix visibility time of `Saved` state.
This fixes the bug where the `Saved` state button faded out almost
instantly (that is actually 300 ms) and `Discard` button fades out
along with `Saved` state button; the key problem here was that the
setTimeout intended to fade was actually delaying the transition from
"saving" to "saved".

Now, first of all, we use `setTimeOut` function to fadeout elements giving
fadeout_delay time as `800 ms` and we hide discard button during `saving`
state. Also, when `Discard` button is selected, `Save changes` and `Dicard`
fade out simultaneously.

Fixes: #11737.
2019-03-10 13:08:57 -07:00
Pragati Agrawal 6a159b5743 settings_org: Refactor `change_save_button_state` function.
This makes the `change_save_button_state` funtion more clear and readable
by removing too many occurences of `.find()` and `.attr()` function.
2019-03-10 13:07:48 -07:00
Puneeth Chaganti abc9a7a4f9 digest: Use a list of Message objects to gather_hot_conversations.
Instead of a queryset of UserMessage objects, `gather_hot_conversations` now
accepts a list of Message objects.
2019-03-09 23:25:26 -08:00
Puneeth Chaganti cb5e9107f4 digest: Directly fetch recipient ids from the DB.
Instead of iterating over Subscriptions and creating the list of home view
recipients, the query now only fetches recipient IDs from the DB.
2019-03-09 23:25:26 -08:00
Harshit Bansal 94649f58f2 tests: Refactor `query_ldap()` and add complete test coverage. 2019-03-09 22:12:51 -08:00
Harshit Bansal b519e6594e management: Move `query_ldap` function to `zproject/backends.py`.
This will make it simpler to organize and unit-test all of our
authentication backend code.
2019-03-09 22:12:36 -08:00
Tim Abbott 873aca4a82 auth: Add detailed comments for auth subsystem.
Now that we've more or less stabilized our authentication/registration
subsystem how we want it, it seems worth adding proper documentation
for this.

Fixes #7619.
2019-03-09 22:08:13 -08:00
Dominik Grybos 0b02ea4e54 docs: Add troublshooting note for macOS Mojave NFS issues.
Fixes #11546.
2019-03-09 16:49:34 -08:00
Tim Abbott fac875a922 docs: Eliminate most development manual install docs.
This documentation had a tendency to bitrot, and in any case now that
we have tooling for doing Fedora (etc.) from provision, it's likely
the case that adding other Linux/UNIX distros we care about to
provision would not be difficult and is a better path than maintaining
this manually-curated duplicate of `tools/provision`.

Further, even if this documentation was maintained, one would still
end up wanting to run `provision` after rebasing a branch, so it was
never particularly practical for extended development.
2019-03-09 16:39:06 -08:00