Commit Graph

3885 Commits

Author SHA1 Message Date
Tim Abbott d8d0492d64 import: Restructure uploads path logic to be more similar.
This is preparation for future deduplication of the two redundant
uploads backends.
2018-12-05 16:15:01 -08:00
Tim Abbott 671ceccd78 import: Deduplicate medium avatars special logic.
This requires a bit of care with upload_backend to avoid breaking how
we mock that class in our tests.
2018-12-05 16:15:01 -08:00
Tim Abbott 36b43a6d7a import: Deduplicate first block of import_uploads logic. 2018-12-05 16:15:01 -08:00
Tim Abbott f80bab58c0 import_realm: Add progress indicator for importing uploads.
This makes it easier to see how we're doing when uploading a very
large number of files.
2018-12-05 16:15:01 -08:00
Greg Price 60de598cb5 user_agent: Use re.X to make the regex a little more readable. 2018-12-05 14:58:13 -08:00
Tim Abbott 380231af9d push_notifications: Add tests for BrokenPipeError case.
This was missing in d723dbfef7.
2018-12-05 10:44:25 -08:00
Tim Abbott b47535d8bb push notifications: Fix exception when handling deleted messages.
If a user deletes message between when it triggered a potential push
notification for another user, and when that notification was actually
sent, we'd end up with a situation where the `Message` table didn't
have an entry for the requested message ID.

The clean fix for this is to still throw an exception in the event
that the message doesn't exist at all, but if it exists in
ArchivedMessage, don't throw a user-facing exception.
2018-12-05 10:38:37 -08:00
Tim Abbott d723dbfef7 push_notifications: Handle BrokenPipeError from APNS.
This seems to happen when Apple is having a partial outage on some of
their APNS shards; it should be treated like other networking errors
connecting to APNS (with an automatic retry).
2018-12-05 10:02:18 -08:00
Anders Kaseorg 9ba860b4f2 Avoid double redirects to /login and then to /login/.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-12-04 16:12:58 -08:00
Tim Abbott 02a79b677b send_email: Extract handle_email_format_changes and use.
Apparently, we have a second code path where we might try to call
send_email library functions on old data, namely in the
queue_processors codebase.  So we apply the same migration logic here.
2018-12-04 16:08:18 -08:00
Eeshan Garg 901100bebf webhooks/jira: Decode topics on our end.
The problem addressed in 006e47198c
resurfaces with topics when a user provides a custom topic in the
webhook URL.
2018-12-03 17:37:36 -08:00
Tim Abbott fa59573a65 mypy: Fix mypy errors in send_email logic.
mypy should be able to figure this out given the xor just above, but
it's not surprising that it doesn't.
2018-12-03 16:30:48 -08:00
Raymond Akornor 92dc3637df send_email: Add support for multiple recipients.
This adds a function that sends provided email to all administrators
of a realm, but in a single email. As a result, send_email now takes
arguments to_user_ids and to_emails instead of to_user_id and
to_email.

We adjust other APIs to match, but note that send_future_email does
not yet support the multiple recipients model for good reasons.

Tweaked by tabbott to modify `manage.py deliver_email` to handle
backwards-compatibily for any ScheduledEmail objects already in the
database.

Fixes #10896.
2018-12-03 15:12:11 -08:00
Tim Abbott 4c1e80032a push notifications: Fix unnecessary warning in development.
This prevents the warning about push notifications not being
registered for from being printed in development environment startup
by default.  In development, that's the expected state, and we don't
need to spam up the output with that notice.
2018-12-03 12:03:56 -08:00
Tim Abbott d96624490e bulk_add_subscriptions: Fix confusing access to user_profile.realm.
Previously, we frequently accessed user_profile.realm from outside the
loops that interact with UserProfile objects.  This variable reuse
outside the loop could be confusing and should be a style/lint
violation.

While in this case, the behavior was correct (in that all users in the
loops were within the same realm), extracting a separate `realm`
variable significantly clarifies what's going on here.
2018-12-03 10:38:10 -08:00
Tim Abbott 5696ad5490 sessions: Suppress coverage error until we can debug. 2018-12-01 10:08:57 -08:00
Tim Abbott 18e2118fc8 Revert "sessions: Stop using the cached_db sessions engine."
This reverts commit 2fa77d9d54.

Further investigation has determined that this did not fix the
password-reset problem described in the previous commit message;
meanwhile, it causes other problems.  We still need to track down the
root cause of the original password-reset bug.
2018-11-30 15:54:32 -08:00
Tim Abbott d0f71881f4 docs: Add detailed documentation on the process for sending messages.
This has long been something missing from our suite of documentation.
2018-11-29 16:25:35 -08:00
Vishnu Ks f3033207b9 emails: Cover all LDAP cases when emailing login details.
This provides a nice user experience for folks where we do know what
their LDAP credentials are.

Though we need to fix #10917 before the content in the email with be
correct.
2018-11-29 12:14:22 -08:00
Rohitt Vashishtha c4e50a34d3 bugdown: Refactor get_user to get_user_by_name.
Also adds a warning against the use of this function.
2018-11-29 10:19:08 -08:00
Hemanth V. Alluri 6b1d724f5c zerver: Add bugdown rendering for text custom profile fields.
This is the first step of letting users use Zulip markdown in their
SHORT_TEXT and LONG_TEXT custom profile fields, so that they can
include emphasis, links, etc.

This doesn't include any frontend logic yet, however.
2018-11-29 09:47:32 -08:00
Anders Kaseorg 1d15d72775 zerver/lib/export.py: Avoid shelling out for cp, rm, ln.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-11-28 17:28:17 -08:00
Rohitt Vashishtha 681368b937 bugdown: Refactor get_possible_mentions_info and related functions.
This commit changes the return type of get_possible_mentions_info  to a
list instead of a dict, thus disposing off the hacky logic of storing
users with duplicate full names with name|id keys that made the code
obfuscated.

The other functions continue to use the dicts as before, however, there
are minor variable changes where needed in accordance with the updated
definition of get_possible_mentions_info.
2018-11-28 14:07:52 -08:00
Tim Abbott e4946dd182 bugdown: Rename full_names to mention_texts.
This is another straight variable rename, which will help clarify the
upcoming commits.
2018-11-28 14:07:23 -08:00
Rohitt Vashishtha ccdf893af7 bugdown: Rename get_full_name_info to get_possible_mentions_info. 2018-11-28 14:04:50 -08:00
Rishi Gupta dd0126ff1b settings: Improve error message when deactivating the last user.
This PR was originally started by Rishi Gupta (see #10383).
2018-11-27 12:49:43 -08:00
Eeshan Garg 7fb674cc58 messages: Add Addressee.for_user_ids().
This commit is a part of our efforts surrounding #9474.
2018-11-27 12:09:41 -08:00
Eeshan Garg 91b90460ee lib/actions: Add helper recipient_for_user_ids().
This function is equivalent to recipient_for_emails, but fetches
user_profiles by IDs, not by emails.

This commit is a part of our efforts surrounding #9474, but is
more primarily geared towards adding support for sending typing
notifications by user IDs.
2018-11-27 11:59:11 -08:00
Eeshan Garg 8ffc437e70 addressee: Raise JsonableError in get_user_profiles().
Previously, get_user_profiles() was split into two functions:

* user_profiles_from_unvalidated_emails, which raised a
  ValidationError upon encountering a non-existent user email.
* get_user_profiles, which caught the ValidationError raised
  by user_profiles_from_unvalidated_emails and raised a
  JsonableError instead.

According to Steve Howell, this complexity is partly a relic
of past refactoring and is unnecessarily heavy. It is better to
just raise JsonableError directly.
2018-11-27 11:59:11 -08:00
Eeshan Garg 68a06abb79 actions: Stop using Addressee's helpers in recipient_for_emails.
recipient_for_emails is used by our typing notifications code.
user_profiles_from_unvalidated_emails is used by our typing
notifications code *and* for sending messages.
user_profiles_from_unvalidated_emails is a part of a larger
framework used by Addressee to validate recipient emails when sending
messages and will eventually need to be removed as we move forward
with #9474. So it makes sense to just inline this function within
recipient_for_emails so that we don't break our typing notifications
code in the future.

This commit is a part of our efforts surrounding #9474.
2018-11-27 11:59:11 -08:00
Tim Abbott adf27aae4c python: Remove now-unnecessary str_utils library.
This library was absolutely essential as part of our Python 2->3
migration process, but all of its calls should be either no-ops or
encode/decode operations.

Note also that the library has been wrong since the incorrect
refactoring in 1f9244e060.

Fixes #10807.
2018-11-27 11:57:54 -08:00
Tim Abbott 6a9e6503b7 python: Remove unnecessary NonBinaryStr TypeVar.
This was a potentially useful value (later superceded by use of Text)
when first introduced, but now that we're on Python 3, it should just
be str.
2018-11-27 11:44:09 -08:00
Raymond Akornor d00b889402 auth: Add an organization reactivation flow with admin confirmation.
This adds a web flow and management command for reactivating a Zulip
organization, with confirmation from one of the organization
administrators.

Further work is needed to make the emails nicer (ideally, we'd send
one email with all the admins on the `To` line, but the `send_email`
library doesn't support that).

Fixes #10783.

With significant tweaks to the email text by tabbott.
2018-11-27 10:58:34 -08:00
Tim Abbott 10e8e2acac push notifications: Tweak text for push notification message.
The previous content made it sound like we were actually sending a
push notification, which could be confusing/alarming in some cases
(see e.g. 9c224ccdd3).  Instead, we make
clear that we're sending it to all clients (which one might correctly
suspect is vacuous in the development environment).
2018-11-27 09:53:44 -08:00
Tim Abbott 7b930124d9 push notifications: Add a logger (default-off in tests).
This should suppress some spammy logging output about push
notifications that we were seeing in a large number of unit tests.
2018-11-27 09:45:45 -08:00
Tim Abbott 38a6003472 push notifications: Improve logging for missing configuration.
While it could make sense to print these logging statements at WARN
level on server startup, it doesn't make sense to do so on every
message (though it perhaps did make sense to do so before more recent
changes added good ways to discover you forgot to configure push
notifications).

Instead, we now just do a WARN log on queue processor startup, and
then at DEBUG level for individual messages.

Fixes #10894.
2018-11-27 09:37:57 -08:00
Tim Abbott beb80219a8 push notifications: Fix a comment typo. 2018-11-27 09:18:35 -08:00
Tim Abbott 3bb0956046 upload: Fix ensure_medium_avatar_image for S3 backend.
Previously, it tried to interact with the wrong path for the original
image.
2018-11-26 20:20:27 -08:00
Marco Burstein 6f569719c9 integrations: Change the truncation marker for long messages.
Change the truncation marker from `...` to `\n[message truncated]`
when receiving messages from the API or through e-mail. Also, update
tests to account for the new change.

Fix #10871.
2018-11-26 11:09:39 -08:00
Vishnu Ks ba640bf89d emails: Don't send day2 email if user already has an account. 2018-11-19 11:17:31 -08:00
Vishnu Ks 3258ce1fa4 emails: Cleanup followup day1 email. 2018-11-19 11:16:36 -08:00
Tim Abbott 541bc0afda sessions: Fix filling memcached caches.
Our recent change in 2fa77d9d54 to
disable the cached_db cache backend broke upgrade-zulip-from-git with
an attributeerror; we fix that by checking the session engine before
trying to access its cache-related attributes.
2018-11-18 20:02:28 -08:00
Tim Abbott 209dd5db67 actions: Add a function for changing realm subdomains.
This is initial work, which will help us establish habits of using a
well-tested approach for renaming a Zulip organization (since as part
of https://github.com/zulip/zulip-mobile/issues/3142, we'll likely
need to make this function do more).
2018-11-15 14:39:14 -08:00
Steve Howell eb4d279bbc Extract get_turtle_message().
This seems like kind of a silly function to extract
to topic.py, but it will theoretically help us sweep
"subject" if we change the DB.

It had test coverage.
2018-11-14 23:24:06 -08:00
Steve Howell cc33e4cd0c digest: Eliminate unneeded queries for hot convos.
We can easily keep track of messages by bucket from the
original loop through messages.
2018-11-14 23:24:06 -08:00
Steve Howell 3091412806 refactor: Use messages in gather_hot_conversations().
We rename stream_messages -> stream_ums, and then we get
messages at the top for the loop that only cares about
messages.
2018-11-14 23:24:06 -08:00
Steve Howell f961408782 digest: Reduce queries using select_related().
We use the message a lot for the query modified
here, so I think it's worth taking the up-front
hit of getting bulkier objects to avoid O(N)
hops back to the database.
2018-11-14 23:24:06 -08:00
Steve Howell 27d79352da topic -> subject: Extract get_topic_from_message_info().
This changes files where it's safe to just assume caller
may use either "topic" or "subject", and we prefer "topic"
but support "subject".
2018-11-14 23:24:06 -08:00
Steve Howell aa4e9cec2d minor: Add a code comment and move two constants. 2018-11-14 23:24:06 -08:00
Steve Howell b666a6bd6c subject -> topic: Fix most of api_test_helpers.
We don't want to change anything that can end
up in docs yet, but for test setup, we can switch
to topic_name.
2018-11-14 23:24:06 -08:00