Commit Graph

9445 Commits

Author SHA1 Message Date
Rohitt Vashishtha f993fdd480 markdown: Add _@**Name** syntax for silent mentions.
These mentions look like regular mentions except they do not
trigger any notification for the person mentioned. These are
primarily to be used when you make a bot take an action and
the bot mentions you, or when you quote a message that mentions
you.

Fixes #11221.
2019-01-16 16:01:06 -08:00
Tim Abbott 5fd43fcf56 zoom: Fix coverage of new zoom error handling code path. 2019-01-16 15:58:26 -08:00
Tim Abbott 1660856bf5 zoom: Fix handling of 201 status codes from zoom API.
Apparently, zoom's API will (sometimes?) return a 201 (not 200)
created in response to the API request to create a call.  We fix this
by using the proper requests check for whether or not the request
failed.
2019-01-16 15:37:06 -08:00
Harshit Bansal 2dc500b11e tests: Fix tests broken in previous commit and add some new ones.
This commit fixes some tests broken in commit
32aa4e02ca.
It also adds some tests for full name mapping in the registration
flow using LDAP.
2019-01-16 09:45:51 -08:00
Harshit Bansal 5f76a65b1d emoji: Make unicode/span emojis more accessible.
This commit adds `aria-label="<title_text>"` and `role="img"` to
the generated HTML.

Fixes: #5975.
2019-01-16 09:07:19 -08:00
Harshit Bansal 32aa4e02ca registration: Fix `full_name` mapping in LDAP registration flow.
We forgot to change this part of codebase when introduced changes
to allow split full name mapping in LDAP.
2019-01-16 08:50:21 -08:00
Anders Kaseorg c0792808e2 deliver_email, deliver_scheduled_messages: Remove lock file.
Tim believes it’s useless.

https://github.com/zulip/zulip/pull/11285#discussion_r248110926

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-01-15 16:56:49 -08:00
Anders Kaseorg 0a8598ce65 feedback: Generate the ticket number from the message ID.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-01-15 16:56:49 -08:00
Tim Abbott dfaa2e481d import: Log a warning when avatars can't be thumbnailed.
This fixes a potential crash in the import tool if a single user has a
broken avatar image.
2019-01-15 16:48:04 -08:00
Anders Kaseorg 25d6107a4f docs: Avoid hardcoded /tmp paths in miscellaneous documentation.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-01-15 16:16:12 -08:00
Anders Kaseorg b7179f8f79 webhooks/zabbix: Don’t write unused /tmp/payload.txt file.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-01-15 16:13:30 -08:00
Anders Kaseorg e5bf0c0a69 event_queue: Avoid hardcoded paths in /var/tmp.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-01-15 16:12:05 -08:00
Anders Kaseorg b37ed12df2 export: Let mkdtemp pick its favorite directory.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-01-15 16:08:48 -08:00
Anders Kaseorg 601b5eb036 export: Avoid hardcoded paths in /tmp.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-01-15 16:05:51 -08:00
Hemanth V. Alluri 7d07dc66fd custom profile fields: Fix error handling for admin clearing values.
This commit fixes an error in the logic for allowing admins to edit any
user's CPF (custom profile field) values. The logic allowing users to
edit their own CPF values is however sound. What happens is that of all
the CPF types, for "choice fields" as well as "URL" and "date fields",
when the value is reset/deleted/cleared by the admin in the Admin UI
(organization settings), the frontend would send a null (empty string)
value to the backend for that custom profile field (as this is, after
all, the new value in this case). This would then triggers the backend
validators to return an error message.

We fix this by using the method check_remove_custom_profile_field_value,
that both code paths (user editing their own CPFs and admin editing a
user's CPF) can call.
2019-01-15 15:29:02 -08:00
Hemanth V. Alluri 716bcad393 custom profile fields: Extract check_remove_user_custom_profile_value.
This moves the logic for deleting the user's custom profile field
value in the remove_user_custom_profile_data view function to a method
named check_remove_user_custom_profile_value in actions.py, so that we
can reuse it in the next commit.
2019-01-15 15:28:16 -08:00
Tim Abbott a92a5f19f0 email_mirror: Handle case where email body is empty.
This provides logging that makes clear this situation is a user error.
2019-01-15 11:30:15 -08:00
Tim Abbott 14b2ed649b email_mirror: Don't email errors for emails missing body type.
This lowers the severity on content type errors to not send spammy
error emails, and instead just log a warning.
2019-01-15 11:30:15 -08:00
Aditya Bansal 4e4c33fee2 reminders: Make reminders setup work only for Streams.
We make this change because setting up reminders in PM's didn't
play really well with our current infrastructure. Basically the
reminder messages from the bot can't appear in the same narrow as
that of a PM between two people and therefore we disable it.
Though we make an exception here where a person wants to set up
reminder for himself.
2019-01-14 12:23:48 -08:00
Aditya Bansal aae90d5d6b deliver_scheduled_messages: Make notification bot deliver reminders.
We do this since we are yet to figure out how the entire realm
internal bots scenerio should work and therefore for the timming
we will use notification bot to deliver the reminders.
2019-01-14 12:11:27 -08:00
Hemanth V. Alluri 58fa142c7d subscriptions: Update the color validator to check_color.
Previously, the subscription color attribute had a validator of
check_string, but this is insufficient. Hence this commit update the
validator used to check_color. Fixes #11268.
2019-01-14 10:16:42 -08:00
Hemanth V. Alluri 1f0d319f9f validators: Add a validator called check_color for hex color codes. 2019-01-14 10:16:42 -08:00
Tim Abbott 0c5f4d43f9 ldap: Avoid spammy log messages from fakeldap as well. 2019-01-14 09:52:25 -08:00
Harshit Bansal 00ca98348a ldap: Shorten the name of logger of `sync_ldap_user_data` command. 2019-01-14 09:50:30 -08:00
Harshit Bansal d2b7a24c26 ldap: Adjust `django_auth_ldap` settings to avoid spammy log messages. 2019-01-14 09:50:30 -08:00
Harshit Bansal 5037b2c5c4 management: Fix a bug in parser of `sync_ldap_user_data` command.
After recent changes for extending the `sync_ldap_user_data` to
update a single user, it was not running without specifying any
arguments.
2019-01-14 09:50:30 -08:00
Eeshan Garg e9e075f6e8 webhooks/pagerduty: Stop sending raw payloads in notifications.
For unsupported or invalid payloads, we should just raise the
UnexpectedWebhookEventType exception and let our logging system
take care of recording the payload that caused the error.
2019-01-13 19:09:11 -08:00
Eeshan Garg a1994122ca webhooks/pagerduty: Refactor and make minor improvements.
This commit improves a couple of things:

* All of the message templates are now at the top, a convention
  we follow in a lot of our webhooks.
* Messages are not prefixed with any emojis. We don't do this in
  any of our other webhooks. Plus, the emojis were outdated.
* Remove some superfluous code.
* Use ```quote <quote goes here> ``` style formatting for
  quoted text instead of the `>` character.
2019-01-13 19:09:11 -08:00
Eeshan Garg 74f0d32a21 webhooks/pagerduty: Support events from webhook v2.
This commit only adds support for the four events that have sample
payloads provided for them on the Pagerduty developer website.
Support for the remaining events will be added in subsequent
commits, as we get access to more sample payloads.
2019-01-13 19:09:11 -08:00
Harshit Bansal 71761bc2da ldap: Add a setting to automatically deactivate non_matching users.
Fixes: #11151.
2019-01-13 19:04:09 -08:00
Harshit Bansal 6797dea6c3 auth: Add tests for `ZulipLDAPUserPopulator`.
Fixes: #11041.
2019-01-13 18:51:50 -08:00
Harshit Bansal 05ad6a357b ldap: Add support for two field mapping of full name.
Tests for `sync_full_name_from_ldap()` are pending and will be added
in a separate commit.

Fixes: #11039.
2019-01-13 18:51:50 -08:00
Harshit Bansal 348f370b79 management: Extract `sync_user_from_ldap()`. 2019-01-13 18:51:50 -08:00
Harshit Bansal 6e20a9a419 ldap: Extract `init_fakeldap()`. 2019-01-13 18:51:50 -08:00
Harshit Bansal e09ef69a8d management: Extend `sync_ldap_user_data` to allow update of a single user. 2019-01-13 18:51:50 -08:00
Harshit Bansal 45d50715c5 uploads: Return a HTTP 400 error for a decompression bomb attack.
Fixes: #11215.
2019-01-13 08:17:24 +00:00
Harshit Bansal 22b67a5207 uploads: Tag error messages for translation. 2019-01-13 06:24:09 +00:00
Vaibhav 4219cc497d tooling: Move confirmation_key view to development only views.
Previously, zerver.views.registration.confirmation_key was only
available in development; now we make that more structurally clear by
moving it to the special zerver/views/development directory.

Fixes #11256.
2019-01-11 12:45:21 -08:00
Vaibhav b94892d633 tooling: Move email_log.py to development only views.
Some urls are only available in the development environment
(dev_urls.py); Corresponding views (here email_log.py) is moved to the
new directory zerver/views/development.

Fixes #11256.
2019-01-11 12:45:02 -08:00
Rohitt Vashishtha b7c5ae7bca dependencies: Upgrade markdown from 2.6.11 -> 3.0.1.
This is a major upgrade, and requires some significant compatibility
work:
* Migrating the pattern-removal logic to use the Registry feature.
* Handling the removal of positional arguments in markdown extensions.
* Handling the removal of safe mode.
2019-01-11 11:40:18 -08:00
Eeshan Garg 73eed42b3c webhooks/clubhouse: Extract story labels from references.
It looks like Clubhouse has moved where they store the label name
from the "actions" list to the "references" list.
2019-01-10 20:00:30 -08:00
Raymond Akornor 254bf4c08f send_email: Add support for passing language into send_future_email.
This adds language paramater to send_future_email. As a result, this
properly internationalizes invitation reminder emails, by passing
correct language into send_future_email.

Fixes #11240.
2019-01-09 17:47:58 -08:00
Anders Kaseorg 2ad1a27b28 create-production-venv: Remove Python 2 support.
We still create a Python 2 virtualenv for thumbor but that’s
separate (/srv/zulip-thumbor-venv from
scripts/lib/create-thumbor-venv).

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-01-09 17:32:05 -08:00
Tim Abbott 4c603990d2 hipchat: Use HTML2Text for the content.
While the result is by no means perfect, it's significantly cleaner
than what we had before this.
2019-01-09 16:59:45 -08:00
Tim Abbott 53436766c1 hipchat: Improve import of public room subscribers.
Now, if you pass an api_key, we'll initialize the public room
subscribers to be whatever they were at the time the import happened.

Also, document the situation on the caveats section.
2019-01-09 16:50:00 -08:00
Tim Abbott 035138dd98 hipchat: Refactor code for building subscriptions.
This moves the filtering of invite-only into the caller, and also
adjusts the indentation.
2019-01-09 16:50:00 -08:00
Tim Abbott c34a8f2e69 hipchat: Fix importing of private messages.
Apparently a stupid typing issue meant that we broke this a few weeks
ago.
2019-01-09 16:50:00 -08:00
Harshit Bansal fbaa497dbb registration: Make an incorrect exception check more precise.
In this case, we should be checking for the precise subclass for
readability.
2019-01-09 10:53:12 -08:00
Harshit Bansal a55e101bef ldap: Allow users to login with just LDAP username.
We had an inconsistent behavior when `LDAP_APPEND_DOMAIN` was set
in that we allowed user to enter username instead of his email in
the auth form but later the workflow failed due to a small bug.

Fixes: #10917.
2019-01-09 10:53:12 -08:00
Steve Howell 475108b784 outgoing webhooks: Set a Zulip-specific user-agent.
This now sets the user-agent to something like:

    ZulipOutgoingWebhook/2.0

(It uses the current ZULIP_VERSION.)

Before this change, the user-agent would be
something like `python-requests/2.18.4`.

Fixes #10741
2019-01-09 10:45:46 -08:00
Tim Abbott a3ac94fa26 management: Support multi-line usage in all management commands. 2019-01-09 10:39:29 -08:00
Mateusz Mandera 3e5f89f2fe email_mirror: Add send_to_email_mirror management command.
Closes #11195. We add a management command to allow us to send emails
to the email mirror directly. The command doesn't require any
configuring of email sending or receiving for the email mirror,
it passes the emails directly using the process_message function.
2019-01-09 10:36:16 -08:00
Eeshan Garg 884b67bea7 bot_lib: Raise exception when PM recipient list is empty.
We need to explicitly check for empty recipient lists in
send_message to ensure that internal_send_huddle_message doesn't
call Addressee.for_private with an empty recipient list.
2019-01-08 16:19:20 -08:00
Eeshan Garg 020f2d0db5 send_message_backend: Add support for recipient user IDs.
Note that create_mirrored_message_users has not been updated to
use user IDs.

Tweaked by tabbott to add a unit test for the huddle case as well.
2019-01-08 16:17:20 -08:00
Eeshan Garg 060de48800 mypy: Accept integer user IDs in handle_deferred_message. 2019-01-08 19:08:49 -03:30
Eeshan Garg 82ace8ccc8 addressee: Add support for user IDs to Addressee.legacy_build().
This commit is a part of our efforts surrounding #9474.
2019-01-08 19:08:49 -03:30
Eeshan Garg b3e8e36a8d messages: Check for empty recipient lists in Addressee.
The check for empty recipient lists (the "message_to" argument)
does not belong in check_message. As we implement support for
sending messages by user IDs (see #9474), we will be extending
much of the existing code in Addressee.legacy_build that validates
recipient lists. Therefore, Addressee.legacy_build is a much more
apt area to check for empty recipient lists.

Also, Addressee.for_private and Addressee.for_user_ids also need
to do their own validation, since not everything goes through
Addressee.legacy_build. It is okay to simply throw a 500 in these
cases because we expect that callers will be doing their own
validation for calls that don't go through Addressee.legacy_build().

This commit is a part of our efforts surrounding #9474.
2019-01-08 19:08:49 -03:30
Eeshan Garg 2f634f8c06 backend: Support user IDs for sending typing notifications.
This is a part of our efforts surrounding #9474.
2019-01-08 19:08:49 -03:30
Eeshan Garg 8f1dba6aad extract_recipients: Support user IDs.
This is a part of our efforts surrounding #9474.
2019-01-08 19:08:49 -03:30
sahil839 4dcccf32f8 zerver: Add feature for notification at rename of a stream.
Feature of sending notification to the stream using notification bot
is added. user_profile is also passed to do_rename_stream for using
the name of user who renamed the stream in notification.
Notification is sent to the stream using
internal_send_stream_message in do_rename_stream.

Fixes #11034.
2019-01-07 19:13:01 -08:00
Tim Abbott 848b2f687c export: Add support for public-streams-only exports.
Previously, this wasn't an explicit feature of the export tool.

Note that the current version still includes metadata on private
streams and private message recipients, just not their messages.
2019-01-07 16:52:02 -08:00
Tim Abbott c6371bb3ef export: Add option to upload exports to S3.
This should make it more convenient to operationalize providing
exports from Zulip Cloud.

Fixes #11178.
2019-01-07 15:28:17 -08:00
Eeshan Garg 539e5e5f41 webhooks/jira: Support comment_created event type.
Fixes: #11200
2019-01-07 14:55:12 -08:00
Tim Abbott 0ffc2ce183 push_notifications: Don't rename import for lxml.html.
This is our standard way of writing code.
2019-01-07 13:15:46 -08:00
Tim Abbott b1cb38e7a0 push_notifications: Remove an unnecessary else clause.
This makes the code a bit cleaner.
2019-01-07 13:15:21 -08:00
Shoumorup 2afdbb139d digest: Fix the styling of /digest page.
This adds a proper template for the /digest page, making it a
reasonable way to view the digest email content for development and
debugging.

Fixes: #11016.
2019-01-07 13:09:29 -08:00
Tim Abbott 828577c3b2 email_mirror: Fix parsing of unicode in subject headers.
Previously, we had some hand-written logic for parsing the subject
line of the email's headers and turning it into a Python string using
each of the valid encodings for an email.  That logic was buggy, and
sometimes resulted in a bytes object being passed into the
`send_zulip`, which would eventually throw an exception.

The fix for this is to use the Python standard library make_header
method for handling internationalized email headers.

https://stackoverflow.com/questions/7331351/python-email-header-decoding-utf-8
2019-01-07 10:21:04 -08:00
Marco Burstein 9ddadd39f4 compose: Add support for using Zoom as the video chat provider.
This adds Zoom call properties to the `Realm` model, creates endpoints
for creating calls, adds a frontend and tests.

Fixes #10979.
2019-01-07 10:00:02 -08:00
Mateusz Mandera 87c95c6f41 email_mirror: Strip RE and FWD from email subject.
Fixes part 1 of #10612. We use a regex to remove RE:, FWD: (and similar
variations) from email subjects. Unit test is included, we add
subjects.json in fixtures containing various subjects to try the
stripping on.
2019-01-05 15:59:19 -08:00
Shubham Dhama e26cba062b settings_invites: Add invited_as column to the table.
This makes it possible to review the roles used in active invitations.
2019-01-05 14:53:09 -08:00
Shubham Dhama efb9128aaa invite: Add option to invite user as guest.
This completes our basic guest user feature.

Fixes: #10818.
2019-01-05 14:52:51 -08:00
Shubham Dhama 42c262b807 invite: Replace `invite_as_admin` usage with `invite_as`.
Since we have already added the `invite_as` field to models, we can now
replace usage of `invite_as_admin` properly with its equivalent `invite_as
== PreregistrationUser.INVITE_AS['REALM_ADMIN']`.

Hence, also removed now redundant `invite_as`.
2019-01-05 14:46:38 -08:00
Shubham Dhama c9414a3572 invite: Rename NORMAL_USER to MEMBER of Preregistration.INVITE_AS.
Renamed to keep the user-facing and the codebase terminologies coherent.
2019-01-05 14:43:06 -08:00
Tim Abbott 2adb0734d1 decorator: Set an error code for invalid API key errors.
This should make it easily for mobile/terminal apps to handle
situations like the user's API key changing.

Also fix the fact we were incorrectly using a 400, not 401, status
code for this case.
2019-01-05 11:29:34 -08:00
Tim Abbott 2ba24bdd09 settings: Fix flushing of API key cache.
The logic for flushing the API key has been broken every since we
added the cache, since we were incorrectly flushing the new API key,
not the old API key, from the cache after regeneration.
2019-01-04 17:26:10 -08:00
Tim Abbott bd5745ab35 test_settings: Extend caching section of API key test. 2019-01-04 17:26:05 -08:00
Tim Abbott 6eda129741 export: Export and import analytics table data.
This should eliminate the need to do manual analytics work when
importing organizations imported/exported using the zulip -> zulip
import/export tools.
2019-01-04 16:22:18 -08:00
Tim Abbott 48ccb3ad18 import: Move realm_tables to the appropriate file.
These had ended up in the wrong place when we split export from
import.
2019-01-04 16:22:18 -08:00
Tim Abbott b33e0ad539 import: Fix pointer logic to sort by message_id.
Previously, the pointer calculation logic wasn't sorting by message
ID, which caused the database queries to not properly use the indexes
they should.
2019-01-04 16:22:18 -08:00
Rishi Gupta c1d40e22a7 emails: Update text for confirm_new_email. 2019-01-04 14:32:21 -08:00
Eeshan Garg ecb506f13c webhooks/clubhouse: Support adding owners to stories. 2019-01-04 14:27:46 -08:00
Tim Abbott b7fb919dfa hipchat: Don't enable slim_mode by default.
For small organizations, generally one prefers the non-slim_mode
default behavior.
2019-01-04 11:30:25 -08:00
Tim Abbott 492230f405 hipchat import: Always include deleted users in import.
The slim_mode setting had been incorrectly configured to skip
"deleted" users, resulting in bugs where private messages with deleted
users would not be imported.
2019-01-04 11:23:02 -08:00
Aditya Bansal 079dfadf1a camo: Add endpoint to handle camo requests.
This endpoint serves requests which might originate from an image
preview link which had an http url and the message holding the image
link was rendered before we introduced thumbnailing. In that case
we would have used a camo proxy to proxy http content over https and
avoid mix content warnings.

In near future, we plan to drop use of camo and just rely on thumbor
to serve such images. This endpoint helps maintain backward
compatibility for links which were already rendered.
2019-01-04 10:27:04 -08:00
Aditya Bansal 3ee69f3da9 thumbnails: Add setting THUMBNAIL_IMAGES.
This setting splits away part of responsibility from THUMBOR_URL.
Now on, this setting will be responsible for controlling whether
we thumbnail images or not by asking bugdown to render image links
to hit our /thumbnail endpoint. This is irrespective of what
THUMBOR_URL is set to though ideally THUMBOR_URL should be set
to point to a running thumbor instance.
2019-01-04 10:27:04 -08:00
Aditya Bansal 26c6ef1834 thumbnails: Fix bug with use of filters in thumbnail generation.
We used to add sharpen filter for all the image sizes whereas it was
intended for resized images only which would have been smoothened
out a bit by the resize operation.
This unnecessary use of the filter used to result in weird issues
with full size images.
For example: Image located at this url:-
http://arqex.com/wp-content/uploads/2015/02/trees.png
When rendered in full size would have just boundaries visible.
2019-01-04 19:06:01 +05:30
Tim Abbott 4eb80bf3b4 decorators: Don't attempt to rate-limit AnonymousUser.
This fixes our support for sending browser errors to the server for
portico pages in production, which previously hit a rate-limiter
exception.
2019-01-03 15:17:50 -08:00
Tom Daff f2e06128c6 email_mirror: Add email address parsing.
When trying to find the email gateway address, use the
`email.util.getaddresses` function to deal with cases
where multiple recipients are included in the email header
or the stream address appears as an angle-addr with a
name given (e.g. if someone added it to their address book).

Added some other headers where the required address may
appear: "Resent" headers are sometimes used for forwarding,
and streams may also be found in CC. There is no way to find
the address if the email was recieved as a BCC.
2019-01-03 13:34:20 -08:00
Tim Abbott e3c8e8a839 models: Add __str__ method for emoji reactions.
This makes it more convenient to print them.
2019-01-03 13:26:25 -08:00
Tim Abbott a1919971e4 import: Handle invalid data-user-id values for mentions.
This is an issue with zulip -> zulip server data imports.
2019-01-02 15:23:09 -08:00
Tim Abbott b63f8b59b2 import: Handle corner case around EMAIL_GATEWAY_BOT emails. 2019-01-02 15:23:09 -08:00
Tim Abbott 8cfea958de import: Fix pointer logic for zulip->zulip imports.
Previously, the pointer was almost guaranteed to be an invalid random
value, because we renumber message IDs unconditionally now.
2019-01-02 15:23:09 -08:00
Tim Abbott c03615b982 import: Fix uploading avatars with S3 upload backend.
This should hopefully be the last commit of this form; ultimately, my
hope is that we'll be able to refactor the semi-duplicated logic in
this file to avoid so much effort going into keeping this correct.
2019-01-02 15:23:09 -08:00
Tim Abbott 1ff3dd0ef7 clubhouse: Don't 500 for unhandled event types. 2019-01-02 15:23:09 -08:00
Tim Abbott ff9714eb32 register_server: Prompt for ToS inside the command.
This way, we don't need to tell new sysadmins how to rerun the script.
2019-01-02 11:19:36 -08:00
Tim Abbott af6f45adab presence: Fix a mypy error with useless result object. 2019-01-02 11:06:20 -08:00
Steve Howell a2614956d5 status: Add away_user_ids to page_params.
(Also, any client that registers for 'user_status'
will see `away_user_ids`.)
2019-01-02 09:16:31 -08:00
Steve Howell 423db23c36 status: Add endpoint to toggle "away" status. 2019-01-02 09:16:30 -08:00
Steve Howell 7d58b05a51 tests: Add capture_event() helper.
For many actions, we make a single call to
send_event, and it's kind of heavy now to
properly assert we made one call, and we
don't need to exercise all the tornado code
to prove that the action was written correctly.
2019-01-02 09:12:10 -08:00
Steve Howell a8301ca14a status: Add UserStatus model and core library for away status. 2019-01-02 09:12:03 -08:00
Hemanth V. Alluri e3aed0f7bc custom profile fields: Markdown rendering for custom profile field values.
This makes it possible it include our standard markdown formatting in
one's custom profile fields, allowing for links, emphasis, emoji, etc.

Fixes #10131.
2019-01-01 21:06:21 -08:00
Hemanth V. Alluri 28d344b4b5 custom profile fields: Pass value as part of a dictionary.
While we're at it, we remove the JSON parsing that was part of the
user field code path, since this function isn't responsible for
rendering user fields.
2019-01-01 21:05:28 -08:00
Steve Howell b4bc6b4445 emails: Fix subject -> email_subject in send_email.py. 2019-01-01 20:49:00 -08:00
Steve Howell 4da28efc90 emails: Fix subject -> email_subject in error_notify.py. 2019-01-01 20:48:59 -08:00
Steve Howell 16328732d4 refactor: Remove "subject" from stream_topic.py. 2019-01-01 20:48:56 -08:00
Tim Abbott 22f39ff8e7 list_realms: Display '' as string_id for root domain realm. 2018-12-30 10:36:18 -08:00
Tim Abbott e48c9245ff list_realms: Fix the --all option.
I don't think anyone's used this in years, but it could be handy.
2018-12-30 10:36:18 -08:00
Hudda 50e14dbe97 list_realms: Print the full domains of the organizations.
This adds a column to display the full domains of realms.  Tweaked by
tabbott to use fewer columns.

Fixes part of #11015.
2018-12-30 10:36:00 -08:00
Shubham Dhama 93e3543e5e user: Extend `do_create_user` for guest users. 2018-12-30 10:16:17 -08:00
Shubham Dhama 235b0db82b preregistrationuser: Add a model field for all type of invited users.
This adds a model field `invited_as` to PreregistrationUser model class
which will replace `invited_as_admin` in future. Intentions behind adding
this are that we can specify "types" of users for an invited person other
than admin or regular that is, guest user or maybe many others in the
future.
2018-12-30 10:16:17 -08:00
Tim Abbott 74ff77d366 import: Always set a valid content-type for S3 backend.
The octet-stream content type is potentially under-specified, but it's
better than potentially submitting None and increases consistency of
this part of the codebase.
2018-12-29 22:13:11 -08:00
Tim Abbott f0c7424957 import: Fix sending floats to boto S3 metadata keys.
The boto library's s3 interface allows setting only string-format
metadata keys.  So we need to cast the last_modified floating-point
timestamp into a string before storing on the S3 object.

This bug mostly broke uploading avatars when using the S3 storage backend.
2018-12-29 22:09:31 -08:00
Tim Abbott 41b7d9a4c8 hipchat: Handle unusual emoticons.json format.
Apparently, hc-migrate can generate emoticons.json files with a
somewhat different format.  Assuming that other files are in the
normal format, we should be able to handle it like this.

See report in #11135.
2018-12-29 18:59:31 -08:00
Tim Abbott 44f117ac72 hipchat: Handle case where emoticons.json is not in export.
Apparently, some methods of exporting from HipChat do not include an
emoticons.json file.  We could test for this using the
`include_emoticons` field in `metadata.json`, but we currently don't
even bother to read that file.  Rather than changing that, we just
print a warning and proceed.  This is arguably better anyway, in that
often not having emoticons.json is the result of user error when
exporting, and it's nice to flag that this is happening.

Fixes #11135.
2018-12-29 18:54:50 -08:00
Tim Abbott 331984c322 ldap: Cast account_control_values to int.
This value will usually apparently come through the LDAP API as a
string, apparently.
2018-12-29 16:35:13 -08:00
Vishnu Ks e5f7d65231 signup: Set oldest account as default option in import settings.
Fixes: #11018
2018-12-29 15:01:09 -08:00
Eeshan Garg d999e79a8b webhooks/bitbucket2: Decode improperly encoded stream names and topics. 2018-12-29 14:51:38 -08:00
Eeshan Garg 31f82705e2 webhooks/bitbucket: Decode improperly encoded stream names and topics. 2018-12-29 14:51:38 -08:00
Tim Abbott a52eeb364b reactions: Avoid 500 when trying to double-add a reaction.
This IntegrityError has been happening occasionally in production due
to races, likely due to some sort of mobile app double-post bug.
Handle this by avoiding a 500, and returning the same 400 we would do
if there hadn't been a race.
2018-12-29 14:10:48 -08:00
Eeshan Garg c4fd0a659f webhooks/statuspage: Update docs, logo and screenshots. 2018-12-29 10:29:16 -08:00
Eeshan Garg 1b539a51de webhooks/opsgenie: Update docs, logo and screenshots.
Note that the word "genie" in Opsgenie is not supposed to be
capitalized.
2018-12-29 10:29:16 -08:00
Eeshan Garg 8bab4ceba4 webhooks/bitbucket2: Update docs, logo and screenshots. 2018-12-29 10:29:16 -08:00
Eeshan Garg 9c584e6bb8 webhooks/trello: Remove superflous Advanced section.
This section is largely unnecessary, doesn't convey any useful
information, and is probably a remnant from an older version of
this doc that we forgot to remove.
2018-12-29 10:29:16 -08:00
Eeshan Garg da663104f3 webhooks/trello: Update logo and screenshots. 2018-12-29 10:29:16 -08:00
Eeshan Garg 776bdab567 webhooks/jira: Update logo and screenshots. 2018-12-29 10:29:16 -08:00
Steve Howell c35afb942f Accept stream_id for muting endpoints. 2018-12-27 20:26:30 -08:00
Eeshan Garg 4a2a3e3640 webhooks/gitlab: Support issues with multiple assignees.
The fixtures added in this commit were generated by Adam Birds.
2018-12-27 20:15:00 -08:00
Eeshan Garg 488f558d49 test_docs: Print better error messages for failed responses. 2018-12-27 20:13:08 -08:00
Eeshan Garg 8a02e177e3 bugdown: Trigger test failure for invalid Markdown include statements.
This commit adds a custom Markdown include extension which is
identical to the original except when a macro file can't
be found, it raises a custom JsonableError exception, which
we can catch and then trigger an appropriate test failure.

Fixes: #10947
2018-12-27 20:13:08 -08:00
Tim Abbott c995e8e2ae import: Ensure presence of basic avatar images for HipChat.
Our HipChat conversion tool didn't properly handle basic avatar
images, resulting in only the medium-size avatar images being imported
properly.  This fixes that bug by asking the import tool to do the
thumbnailing for the basic avatar image (from the .original file) as
well as the medium avatar image.
2018-12-27 17:47:09 -08:00
Rishi Gupta 955dfc6397 billing: Remove references to Realm.has_seat_based_plan. 2018-12-24 10:27:56 -08:00
Rishi Gupta e7220fd71f billing: Do subscription management in-house instead of with Stripe Billing.
This is a major rewrite of the billing system. It moves subscription
information off of stripe Subscriptions and into a local CustomerPlan
table.

To keep this manageable, it leaves several things unimplemented
(downgrading, etc), and a variety of other TODOs in the code. There are also
some known regressions, e.g. error-handling on /upgrade is broken.
2018-12-22 13:39:30 -08:00
Vishnu Ks bcae08470c emails: Use ugettext instead of ugettext_lazy in signals.
ugettext_lazy was printing the object in the email template
instead of the string.
2018-12-20 19:35:23 -08:00
Rishi Gupta bf8db6f634 emails: Update text for confirm_registration. 2018-12-20 16:26:19 -08:00
Rishi Gupta 0844c9a6ec emails: Update text for password_reset.
Note that a pretty common use case for this is a realm admin sending this to
everyone after an import from HipChat or Slack. So this adds the realm_name
to the title (so that there is something they might recognize) and kept the
wording generic enough to accommodate the user not having clicked anything
to get this email.

Also strengthens the tests a bit to better test the complicated template
logic.
2018-12-20 16:26:19 -08:00
Rishi Gupta 2da0cb7a44 emails: Update text of realm_reactivation. 2018-12-20 16:26:19 -08:00
Rishi Gupta fb56720054 emails: Update text for invitation and invitation_reminder. 2018-12-20 16:26:19 -08:00
Rishi Gupta d8e16143d4 emails: Clean up code surrounding the call to find_team. 2018-12-20 16:26:19 -08:00
Rishi Gupta 292683f0d5 emails: Add a few more emails to email_log.py. 2018-12-20 16:26:19 -08:00
Rishi Gupta abbe7b8f65 context_processors: Add user_name to common_context. 2018-12-20 16:26:19 -08:00
Vishnu Ks 6d29525ef0 emails: Fix broken email revert process in email_log. 2018-12-19 11:24:57 -08:00
Tim Abbott 9c3f38a564 docs: Automatically construct OpenAPI metadata for help center.
This is somewhat hacky, in that in order to do what we're doing, we
need to parse the HTML of the rendered page to extract the first
paragraph to include in the open graph description field.  But
BeautifulSoup does a good job of it.

This carries a nontrivial performance penalty for loading these pages,
but overall /help/ is a low-traffic site compared to the main app, so
it doesn't matter much.

(As a sidenote, it wouldn't be a bad idea to cache this stuff).

There's lots of things we can improve in this, largely through editing
the articles, but we can deal with that over time.

Thanks to Rishi for writing all the tests.
2018-12-19 10:18:20 -08:00
Vishnu Ks 78fea79c06 emails: Simplify new_login device info section. 2018-12-19 09:21:09 -08:00
seresheim 49dbd85a89 auth: Add support for Azure Active Directory authentication.
This takes advantage of all of our work on making the
python-social-auth integration reusable for other authentication
backends.
2018-12-18 16:39:03 -08:00
Tim Abbott 9d058f9193 test_auth_backends: Eliminate manual lists of authentication backends.
This should dramatically reduce the manual work involved with
correctly adding a new authentication backend to Zulip with this test
suite.
2018-12-18 16:36:45 -08:00
Tim Abbott 11fcbe52a6 auth: Automate calculation of get_auth_backends_data.
This change lets us eliminate the need for new authentication backends
to edit get_auth_backends_data, since we're just computing it from the
official registry in zproject/backends.py.  Should save a few lines of
work whenever we add a new auth backend, and make that more accessible
to new contributors.
2018-12-18 16:23:21 -08:00
Tim Abbott bd0596e711 context_processors: Automated oauth backend enabled logic. 2018-12-18 16:23:21 -08:00
Eeshan Garg 6d51c065d3 webhooks/gitlab: Add support for confidential issues and notes.
With some minor changes by Eeshan Garg.

Fixes #10478.
2018-12-18 14:37:44 -08:00
Eeshan Garg 124cb9cca3 webhooks/zabbix: Handle payloads with missing data properly. 2018-12-18 13:44:35 -08:00
Tim Abbott 3c70a61132 models: Deduplicate definitions of push tokens.
Since c57c4cf703, these are no longer
different implementations, so we can deduplicate them.  Because Django
is smart, no migration is required.
2018-12-18 13:05:23 -08:00
Eeshan Garg a56dbdb0f3 webhooks/clubhouse: Ignore requests with empty request body. 2018-12-18 12:54:04 -08:00
Eeshan Garg dedf56bf31 webhooks/clubhouse: Support story/epic deletion. 2018-12-18 12:54:04 -08:00
Joshua Pan ad1df0ebeb settings: Add support for customizing the top-left logo.
This adds a new realm_logo field, which is a horizontal-format logo to
be displayed in the top-left corner of the webapp, and any other
places where we might want a wide-format branding of the organization.

Tweaked significantly by tabbott to rebase, fix styling, etc.

Fixing the styling of this feature's loading indicator caused me to
notice the loading indicator for the realm_icon feature was also ugly,
so I fixed that too.

Fixes #7995.
2018-12-18 12:44:52 -08:00
Eeshan Garg ebf5104923 bot_lib: Add user_id attribute to EmbeddedBotHandler.
Support for extended mention syntax was added as a part of
commit fbe99b812ee8fbca7257a5b7156c57a6cd74195b in the
python-zulip-api repository. The relevant function,
extract_query_without_mention now relies on the client's ID
in order to check for the extended syntax. Since the
EmbeddedBotHandler has no user_id attribute, the latest
python-zulip-api release broke a test in the main repo.
2018-12-17 18:37:20 -03:30
Tim Abbott d469c14d11 docs: Advertise arbitrary org GitHub patterns for linkifiers.
Now that we support this, we should make it really obvious that we do.

Also, link to the Help Center article.
2018-12-17 12:58:52 -08:00
Tim Abbott c87a533b3b realm filters: Make validation error messages more useful. 2018-12-17 12:25:12 -08:00
Tim Abbott 34e39248fc linkifiers: Add validation support for multiple items.
This is a simple change to our validation, to allow multiple copies of
the main linkifier syntax, which lets us support things like generic
GitHub URLs.

Fixes #10914.
2018-12-17 12:19:59 -08:00
Tim Abbott a0da4f6d30 python: Clean up various if False blocks.
Most of these are now-unnecessary typing imports; some are just
improved comments for those with other mypy motivations.
2018-12-17 11:14:47 -08:00
Tim Abbott b29d66f34b i18n: Rename email subject files so i18n detects them.
Apparently, when we renamed these files to no longer have a .txt
extension, we accidentally removed them from the set of strings for
translation, because `manage.py makemessages` by default only
processes .txt and .html files under the templates/ directory.

Fix this by adding a .txt extension.
2018-12-17 10:00:49 -08:00
Tim Abbott b2fc017671 i18n: Use the recipient's language when sending outgoing emails.
It appears that our i18n logic was only using the recipient's language
for logged-in emails, so even properly tagged for translation and
translated emails for functions like "Find my team" and "password
reset" were being always sent in English.

With great work by Vishnu Ks on the tests and the to_emails code path.
2018-12-17 09:49:36 -08:00
Vishnu Ks b10c23c233 emails: Call send_email when DEVELOPMENT_LOG_EMAILS is enabled.
The previous version was also doing almost the same thing.
But checking for DEVELOPMENT_LOG_EMAILS would allow us
to control the call of send_email by altering the value
of DEVELOPMENT_LOG_EMAILS in tests.
2018-12-17 09:40:32 -08:00
Tim Abbott e9f889d57e portico: Fix totally broken ToS re-accept form styling.
This form isn't actively used, which is how it ended up broken, but it
basically didn't display its content properly at all.

Convert it to use our standard white-box framework.

This still doesn't look great in various ways, but it's at least not
obviously totally busted now.
2018-12-17 07:28:29 -08:00
Tim Abbott d005f4edd3 management: Fix incorrect create_user docsting on ToS. 2018-12-16 22:23:35 -08:00
Tim Abbott d9912cb23c management: Fix buggy create_user management command.
The create_user management command incorrectly was double-calling
notify_new_user.
2018-12-16 22:22:52 -08:00
Tim Abbott 8a90441d2f slack import: Import long-inactive users as long-term idle.
This avoids creating UserMessage rows for long-inactive users in
organizations with many thousands of users.
2018-12-16 18:52:20 -08:00
Tim Abbott 37189e1f9d soft deactivation: Handle case where a user has no message history.
I'm aware of at least one case where this happened with some imported
data history; better to not have that crash.
2018-12-16 18:52:20 -08:00
Tim Abbott f47f263655 soft deactivation: Avoid giant transaction.
The previous logic for soft deactivation ended up doing a giant
transaction in the case that there were thousands of users to
deactivate; this was messy and potentially buggy.

The batched transactions were useful for RealmAuditLog management,
however.  So the right solution is to do reasonably sized batches
(e.g. 100 users).
2018-12-16 18:52:19 -08:00
Tim Abbott 430ed061c1 migrations: Fix do_batch_update for postgres 10.
Apparently, our do_batch_update method (used, e.g., in a pgroonga
migration) was using semi-invalid syntax that was removed in postgres
10.

Thanks to Ilya Evseev for the report.

Fixes #11063.
2018-12-16 17:46:11 -08:00
Tim Abbott 142d9cb63f report: Allow error-reporting views from unauthed users.
This should make it possible for blueslip error reports to be sent on
our logged-out portico pages, which should in turn make it possible to
debug any such issues as they occur.
2018-12-16 15:44:48 -08:00
Tim Abbott 32f24bff8d stripe: Fix exception handling for suppressed event types. 2018-12-15 14:14:09 -08:00
Tim Abbott 930e65d1be push: Include type in add-push-notification events.
This should make us able to clean up the logic for this in the future
(right now, we still need to do the .get() for backwards compatibility).
2018-12-15 13:58:52 -08:00
ishanrai05 4105fb683b notifications: Optimize push notifications code path in tests.
This checks if push_notification_enabled() is set to false in
handle_push_notification and adds an early return statement.

This is a significant performance optimization for our unit tests
because the push notifications code path does a number of database
queries, and this migration means we don't end up doing those queries
the hundreds of times we send PMs or mentions in our tests where we're
not trying to test the push notifications functionality.

This should also have a small message sending scalability improvement
for any Zulip servers without push notifications enabled.

Tweaked by tabbott to fix a few small issues.

Fixes #10895.
2018-12-15 11:12:43 -08:00
Tim Abbott a63eae48cc test_push_notifications: Fix leak that can leak to test flakes.
While reviewing #11012, I discovered a nondeterministic result for
test_signup, which I tracked down to specifically this triple of tests
failing when run in this order:

test-backend GCMSuccessTest \
  zerver.tests.test_push_notifications.TestAPNs.test_get_apns_client \
  zerver.tests.test_signup.LoginTest.test_register

with a query count mismatch like this:

expected length: 73
actual length: 79

Comparing the list of queries, it's clear that test_register was
seeing `push_notifications_enabled()` returning True in this test order.

It's not clear why GCMSuccessTest was required here (it was!), but
further debugging determined the problem was that
`test_get_apns_client` left the _apns_client initialization system in
a state where get_apns_client would return a non-None value, resulting
in push_notifications_enabled() returning True for future tests.

The immediate fix is to just reset the `_apns_client` and
`_apns_client_initializedstate` state properly after the test runs;
but arguably we should do a larger refactor to make this less
fragile.
2018-12-15 11:12:43 -08:00
Rishi Gupta 111eda604b portico: Add /atlassian for Zulip users migrating from HipChat. 2018-12-14 23:42:47 -08:00
Roger Souza 69da22d998 api docs: Document the custom emoji upload endpoint.
Tweaked by tabbott to fix some English phrasing and make the file
pointer thing require less Python knowledge.

Fixes: #10746.
2018-12-14 14:19:28 -08:00
Tim Abbott 67981725ec send_email: Fix migration code path for ScheduledEmail.
The previous migration code path was broken in two ways:

* ScheduledEmail objects generally contain a `None` value for
  whichever of `to_user_id` and `to_email` isn't in use; this could
  result in us sending a [None] to send_email(), which doesn't make
  sense.

* We were calling handle_send_email_format_changes in the wrong order
  with respect to the JSON loading process.

Thanks to Tom Daff for the report!
2018-12-14 12:46:31 -08:00
Tim Abbott 8c30c36006 realm filters: Expand set of characters allowed in prefixes.
Our list of allowed characters in realm filter patterns has long been
too string; fix this by extending the pattern.

Also, extend the tests to have examples of actual strings one would
use with the patterns, for clarity.

Fixes #10953, fixes #6835.
2018-12-14 11:33:13 -08:00
Tim Abbott 626e191201 ldap: Add support for automatic user deactivation/reactivation.
As part of this, extend our documentation on synchronizing data from
Active Directory.
2018-12-13 16:24:15 -08:00
Tim Abbott 81271b0d20 sync_ldap_user_data: Process deactivated users as well.
Technically, we will only need to process deactivated users for the
purpose of reactivating them (and can ignore, e.g., name changes).
But it's simplest to just process them unconditionally.
2018-12-13 16:24:15 -08:00
Tim Abbott 0a5221a819 ldap: Extract dev_ldap_directory.py.
This gets what is fundamentally unit testing code out of backends.py.
2018-12-13 16:24:15 -08:00
Tim Abbott 5dd646f33f ldap: Add support for syncing avatar images from LDAP.
This should make life a lot more convenient for organizations that use
the LDAP integration and have their avatars in LDAP already.

This hasn't been end-to-end tested against LDAP yet, so there may be
some minor revisions, but fundamentally, it works, has automated
tests, and should be easy to maintain.

Fixes #286.
2018-12-13 13:39:22 -08:00
Rishi Gupta 8a95526ced billing: Always transition to Realm.LIMITED via do_change_plan_type.
Fixes a bug in import_realm where secondary attributes like message
visibility weren't being set, and also makes bugs like this less likely in
the future.

Also, putting the plan_type change at the end of import_realm, so that
future restrictions to LIMITED realms don't affect the import process.
2018-12-13 13:26:24 -08:00
Rishi Gupta b245c661da billing: Change do_change_plan_type to take a realm instead of a user.
More often than not, changes in plan type are not directly due to user
action.
2018-12-13 13:26:24 -08:00
Tim Abbott e7746809d2 stripe: Fix exception handler for suppressed events.
Apparently, we incorrectly placed the try/except block around the
common code, not the code that can actually raise these exceptions.
2018-12-13 10:22:19 -08:00
Tim Abbott a6ca95dfc4 slack import: Fix all messages being imported to one channel.
This was an ugly variable-escape-from-loop regression introduced in
e59ff6e6db.
2018-12-12 17:54:37 -08:00
Vishnu Ks 8a1794caa3 message: Store the value of first_visible_message_id in Realm table.
This eliminates a bunch of potentially buggy caching code, with no
material negative side effects.
2018-12-12 15:11:17 -08:00
Tim Abbott ee901ac8b1 django: Remove the Django contrib Sites app to fix 2FA QR codes.
Apparently, Django's get_current_site function (used, e.g., in
django-two-factor to look up the domain to use in QR codes) first
tries to use the Sites framework, and if unavailable, does the right
thing (namely, using request.get_host()).

We don't use the Sites framework for anything in Zulip, so the correct
fix is to just remove it.

Fixes #11014.
2018-12-12 15:01:55 -08:00
Tim Abbott d6217eb862 slack import: Fix empty values for custom profile fields.
The Slack import process would incorrectly issue
CustomProfileFieldValue entries with a value of "" for users who
didn't have a given CustomProfileField (especially common for the
"skype" and "phone" fields).  This had no user-visible effect, but
certainly added some clutter in the database.
2018-12-12 12:58:27 -08:00
Tim Abbott 5d5b1afd2d rate_limiter: Improve handling of deadlocks.
We should rate-limit users when our rate limiter deadlocks trying to
increment its count; we also now log at warning level (so it doesn't
send spammy emails) and include details on the user and route was, so
that we can properly investigate whether the rate-limiting on the
route was in error.
2018-12-12 12:19:01 -08:00
Tim Abbott 8a11c94a2d fakeldap: Add thumbnailPhoto/jpegPhoto for testing avatar syncing. 2018-12-12 11:23:33 -08:00
Tim Abbott e9900b2bdf gitter: Do something reasonable with invalid fullnames. 2018-12-12 10:07:52 -08:00
Tim Abbott 27f3919259 push_notifications: Also treat ConnectionError as a retry.
It's a bug that the HTTP20 library isn't catching these, but we just
have to work around it.
2018-12-12 09:40:51 -08:00
Tim Abbott 914ae88c62 push_notifications: Expand APNS main try/except block.
We should be catching exceptions trying to send the notice as well as
in the result section.
2018-12-12 09:40:51 -08:00
Roman Godov 9c8eeaed85 digest_email: Add endpoint for rendering digest to the web.
Adds "/digest/" endpoint for rendering content of digest email
to the web.

Fixes #9974
2018-12-11 13:38:30 -08:00
Tim Abbott 15d4b71e2e decorator: Skip rate limiting when accessing user uploads.
The code paths for accessing user-uploaded files are both (A) highly
optimized so as to not require a ton of work, and (B) a code path
where it's totally reasonable for a client to need to fetch 100+
images all at once (e.g. if it's the first browser open in a setting
with a lot of distinct senders with avatars or a lot of image
previews).

Additionally, we've been seeing exceptions logged in the production
redis configuration caused by this code path (basically, locking
failures trying to update the rate-limit data structures).

So we skip running our current rate limiting algorithm for these views.
2018-12-11 12:43:21 -08:00
Tim Abbott b7127a395b server_settings: Use a reasonable default when User-Agent not specified.
Apparently, some Zulip clients don't send a User-Agent in their
requests to /server_settings, and this makes that not 500.
2018-12-11 11:30:11 -08:00
Tim Abbott 2a39852e91 auth: Add logging for which client does /server_settings requests.
This is generally helpful for debugging.
2018-12-11 11:30:11 -08:00
Tim Abbott 88368996fb decorator: Rename remote_server_request to skip_update_user_activity.
This makes it reasonable to call process_client from other contexts,
like /server_settings, where we don't have an authenticated
UserProfile object.
2018-12-11 11:30:11 -08:00
Tim Abbott 14bfa74069 registration: Enable copying profile settings in production.
Now that we've styled this feature properly, this makes it possible to
copy various user-preferences type profile data in production when
making a new account with the same email address as an existing
account.
2018-12-10 16:55:07 -08:00
Max Nussenbaum 536de666df register: Style avatar that shows when importing settings.
This styles the avatar and username that show when the registering
user is importing their settings from an existing Zulip account.

Tweaked by tabbott to fix the test/linter failures, a bit of styling,
and tag strings for translation.
2018-12-10 15:40:30 -08:00
Jack Zhang be9b6a6dee compatibility: Add a compatibility check to api_get_server_settings.
This should make it convenient for the mobile app to present errors of
the form "Your Zulip app is not new enough for this Zulip server".
2018-12-10 15:06:09 -08:00
Jack Zhang 8711fe7bae compatibility: Respond with HTTP 400 when no User-Agent header.
Prior to this commit, a request to /compatibility without a User-Agent
header would result in the server 500ing.
2018-12-10 15:04:34 -08:00
Tim Abbott a1d99adce7 auth: Handle SSO_APPEND_DOMAIN in remote_user SSO for mobile.
Apparently, while the main code path through
login_or_register_remote_user was correctly calling
remote_user_to_email(username) to get a proper email address for
situations where auth username != email (i.e. when SSO_APPEND_DOMAIN
is set), we neglected to do so in the mobile_flow_otp corner case.

Fixes #11005.
2018-12-10 10:36:15 -08:00
neiljp (Neil Pilgrim) 63768858ff dependencies: Upgrade mypy to version 0.650.
Broaden the type of the AbstractEnum __reduce_ex__ parameter to object; this
matches the parameter type specified in the latest enum.pyi file in typeshed.

Fixes #10996.
2018-12-09 22:20:12 -08:00
Marco Burstein ba46dc83c6 notifications: Add a setting for changing the notification sound.
Also, add a new notification sound, "ding". It comes from
https://freesound.org, where the original Zulip notification sound comes
from as well. In the future, new sounds can be added by adding audio
files to the `static/audio/notification_sounds` directory.

Tweaked significantly by tabbott:
* Avoided removing static/audio/zulip.ogg, because that file is
  checked for by old versions of the desktop app.
* Added a views check for the sound being valid + tests.
* Added additional tests.
* Restructured the test_events test to be cleaner.
* Removed check_bool_or_string.
* Increased max length of notification_sound.
* Provide available_notification_sounds in events data set if global
  notifications settings are requested.

Fixes #8051.
2018-12-09 21:25:30 -08:00
rht a1ff44a230 refactor: Add a helper function to create s3 buckets.
This refactor makes upgrading boto to boto3 easier.
Based on 43d2f6286c
2018-12-07 13:58:11 -08:00
rht a0dbcde063 export_files_from_s3: Move saving s3 object to local file to a separate function.
This refactor makes upgrading boto to boto3 easier.
Based on 24bf813e8a
2018-12-07 11:37:46 -08:00
rht 0ddb242583 export_files_from_s3: get s3 object info in dict to a separate function.
This refactor makes upgrading boto to boto3 easier.
Based on 24bf813e8a
2018-12-07 11:37:46 -08:00
rht 1cecf0f142 export_files_from_s3: Move checking for s3 oject's metadata to a separate function.
This refactor makes upgrading boto to boto3 easier.
Based on 24bf813e8a
2018-12-07 11:37:46 -08:00
Joshua Pan 441dca0eaf email: Support delivery email in email change system. 2018-12-06 17:43:26 -08:00
Tim Abbott 5c2e7a169f test_events: Use a better assertion when comparing event count. 2018-12-06 17:43:26 -08:00
Tim Abbott 7e6f30caab actions: Extract send_user_email_update_event.
This will make it easy for us to reuse this event-sending logic for
the email_address_visibility change code path.
2018-12-06 16:26:46 -08:00
Tim Abbott e603237010 email: Convert accounts code to use delivery_email.
A key part of this is the new helper, get_user_by_delivery_email.  Its
verbose name is important for clarity; it should help avoid blind
copy-pasting of get_user (which we'll also want to rename).
Unfortunately, it requires detailed understanding of the context to
figure out which one to use; each is used in about half of call sites.

Another important note is that this PR doesn't migrate get_user calls
in the tests except where not doing so would cause the tests to fail.
This probably deserves a follow-up refactor to avoid bugs here.
2018-12-06 16:21:38 -08:00
Tim Abbott 9d32857c37 realm: Add new Realm.email_address_visibility field.
This is preparatory work for settings controlling who can see user
emails; it includes the API-level support for editing it, but no code
to actually enforce the policy.
2018-12-06 15:56:30 -08:00
Rishi Gupta 82ee496efc webhooks/stripe: Update fixture for customer_subscription_created. 2018-12-06 11:29:16 -08:00
Rishi Gupta 76aebac568 webhooks/stripe: Format timestamps as dates. 2018-12-06 11:29:16 -08:00
Rishi Gupta 027d5e90c5 webhooks/stripe: Update how we handle updated events.
Also more explicitly ignores the Stripe Connect related events in the
'account' category.
2018-12-06 11:29:16 -08:00
Rishi Gupta 5fb683e788 webhooks/stripe: Use exceptions instead of json_success to exit main body. 2018-12-06 11:29:16 -08:00
Tim Abbott fc1c146d31 export: Remove assertion on current working directory.
This command hasn't made deep assumptions about CWD for a long time,
and this enables users to run it through a symlink (etc.).

Fixes #10961.
2018-12-06 11:05:40 -08:00
Tim Abbott 9da4b58c8e docs: Fix /help/ sidebar being accidentally missing.
Apparently, we didn't have a test for this, and it was broken in a
recent refactoring.
2018-12-06 10:13:51 -08:00
Vishnu Ks 18f90754da portico: Change wording of finding organization in /accounts/go. 2018-12-06 09:24:22 -08:00
Tim Abbott 460249704e send_email: Fix cleanup for incorrectly migrated jobs.
This logic, unfortunately, had a typo.
2018-12-05 16:30:28 -08:00
Tim Abbott 1adc40f014 import: Deduplicate functions for uploading to S3/files.
We've had a long stream of bugs existed because only one of these two
code paths was tested (usually the local uploads backend).  By
deduplicating these functions, we ensure that this category of bugs no
longer happens.

Following my recent refactor, this is just a straightforward merge,
with code for one or the other backend ending up inside an if
statement.
2018-12-05 16:15:01 -08:00
Tim Abbott c9b801efde import: Use the s3_path attribute for path_maps unconditionally.
While the s3_path is almost always the same as the path, structurally,
`path` is the location in the export object, whereas s3_path is the
URL path.
2018-12-05 16:15:01 -08:00
Tim Abbott f4c5a45f4f import: Fix S3 paths for imported avatar PNG.
Previously, we were incorrectly importing avatar PNGs to a filename
without the .png extension, resulting in them effectively not being
imported.

This was mitigated by the fact that we imported the originals and ran
the appropriate `ensure_` functions, but still a bug.
2018-12-05 16:15:01 -08:00
Tim Abbott 412dc8dcda import: Set last_modified in import_uploads_local.
This has no effect other than to make the S3 and local code paths more
nearly identical.
2018-12-05 16:15:01 -08:00
Tim Abbott d8d0492d64 import: Restructure uploads path logic to be more similar.
This is preparation for future deduplication of the two redundant
uploads backends.
2018-12-05 16:15:01 -08:00
Tim Abbott 671ceccd78 import: Deduplicate medium avatars special logic.
This requires a bit of care with upload_backend to avoid breaking how
we mock that class in our tests.
2018-12-05 16:15:01 -08:00
Tim Abbott 36b43a6d7a import: Deduplicate first block of import_uploads logic. 2018-12-05 16:15:01 -08:00
Tim Abbott f80bab58c0 import_realm: Add progress indicator for importing uploads.
This makes it easier to see how we're doing when uploading a very
large number of files.
2018-12-05 16:15:01 -08:00
Tim Abbott 9e06ab45bf webhooks: Fix HttpResponse with notify_bot_owner_on_invalid_json.
Apparently, there was a bug in notify_bot_owner_on_invalid_json, where
we didn't reraise the JsonableError.

We fix this with a refactoring that makes the exception layering
clearer as well.
2018-12-05 15:17:31 -08:00
Tim Abbott 9de1bd44e2 compatibility: Clean up linter logic for too-old client. 2018-12-05 15:04:21 -08:00
Greg Price 32c6b81b3a compatibility: Add test data from real server logs.
In a quick scan of today's nginx logs on chat.zulip.org, there
were 20 distinct user-agents that begin with 'ZulipMobile/'.
Here's a representative sampling of them, such that the rest
were all boringly similar to one of these.

First, to make room for these without an excess of copy-paste and
overlong lines, convert this test to a data-oriented style.  The
existing, synthetic cases appear in the new data followed by the
seen-in-the-wild cases.

Happily, the code being tested passes all these new cases unchanged.
2018-12-05 14:58:13 -08:00
Greg Price fb7bfbe9ab compatibility: Require version 16.2.96+ for our Android app.
This release is from 2018-08-22, a little over 100 days ago.

It was the first release with the important fix so that when the
server advises it to stop displaying a notification because the user
has read the message (as the SEND_REMOVE_PUSH_NOTIFICATIONS server
setting enables), the app doesn't instead replace the notification
with a broken one reading "null".  We have that setting running now
on chat.zulip.org, and intend to roll it out more broadly soon.

The `# take 0` thing is a slightly absurd workaround for the fact
that our funky out-of-line way of marking lines to ignore doesn't
work right if there are multiple such lines in a given file that
are equal modulo leading and trailing whitespace.
2018-12-05 14:58:13 -08:00
Greg Price 0fa4fdcff9 compatibility: Rewrite verbose test for compact legibility.
This will facilitate adding a bunch more test cases shortly.
2018-12-05 14:58:13 -08:00
Greg Price 2bac6de6c6 compatibility: Add and test a helper find_mobile_os. 2018-12-05 14:58:13 -08:00
Greg Price 1859faccb7 compatibility: Treat empty version as unknown, not old. 2018-12-05 14:58:13 -08:00
Greg Price ebfa4b746d compatibility: Make 1.2 < 1.2.3; rewrite version_lt internals. 2018-12-05 14:58:13 -08:00
Greg Price 557aca2aa7 compatibility: Implement a version comparator. 2018-12-05 14:58:13 -08:00
Greg Price 60de598cb5 user_agent: Use re.X to make the regex a little more readable. 2018-12-05 14:58:13 -08:00
Tim Abbott cfeb87c1c9 tornado: Require non-negative lifespan_secs.
Previously, our validation for this field only checked it was an
integer, and you could in theory send invalid negative values here.
2018-12-05 14:50:37 -08:00
Tim Abbott 8e4d6fa045 event_queue: Rename IDLE_EVENT_QUEUE_TIMEOUT_SECS.
This is a default value, not an always-used value, and its name should
reflect that.
2018-12-05 14:48:40 -08:00
Tim Abbott 94dfff1c4e event queue: Don't set a minimum for lifespan_secs.
This makes it more convenient for developers to set very short values
for this (e.g. 1 minute) for the purposes of testing/debugging; there
aren't obvious problems with letting users set short values for this.
2018-12-05 14:47:53 -08:00
Tim Abbott a3c2d49f0c event_queue: Change garbage-collection frequency to 1 minute.
This is designed to help make it more convenient to do manual testing
where we need event queues to be garbage-collected.
2018-12-05 14:42:53 -08:00
Tim Abbott 6dd69b9bff event_queue: Rename ClientDescriptor.idle to expired.
This better reflects the situation with these event queues -- they're
not idle, they are expired and to be garbage collected.
2018-12-05 14:42:53 -08:00
Tim Abbott 408af032a0 event_queue: Remove queue_timeout migration code from 2013.
There's never going to be an event queue without a queue_timeout
property anymore.
2018-12-05 14:24:38 -08:00
rht e59ff6e6db slack import: Eliminate need to load all messages into memory.
This works by yielding messages sorted based on timestamp.  Because
the Slack exports are broken into files by date, it's convenient to do
a 2-layer sorting process, where we open all the files for a given
day, and then sort their messages by timestamp before yielding them.

Fixes #10930.
2018-12-05 12:20:50 -08:00
Tim Abbott 380231af9d push_notifications: Add tests for BrokenPipeError case.
This was missing in d723dbfef7.
2018-12-05 10:44:25 -08:00
Tim Abbott b47535d8bb push notifications: Fix exception when handling deleted messages.
If a user deletes message between when it triggered a potential push
notification for another user, and when that notification was actually
sent, we'd end up with a situation where the `Message` table didn't
have an entry for the requested message ID.

The clean fix for this is to still throw an exception in the event
that the message doesn't exist at all, but if it exists in
ArchivedMessage, don't throw a user-facing exception.
2018-12-05 10:38:37 -08:00
Tim Abbott d723dbfef7 push_notifications: Handle BrokenPipeError from APNS.
This seems to happen when Apple is having a partial outage on some of
their APNS shards; it should be treated like other networking errors
connecting to APNS (with an automatic retry).
2018-12-05 10:02:18 -08:00
Tim Abbott 196ec78261 logging_handlers: Fix unnecessary broken import. 2018-12-04 18:18:07 -08:00
Jack Zhang c26f5c6ee3 Rename `check_compatibility` to `check_global_compatibility`. 2018-12-04 17:56:12 -08:00
Anders Kaseorg 9ba860b4f2 Avoid double redirects to /login and then to /login/.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-12-04 16:12:58 -08:00
Tim Abbott 02a79b677b send_email: Extract handle_email_format_changes and use.
Apparently, we have a second code path where we might try to call
send_email library functions on old data, namely in the
queue_processors codebase.  So we apply the same migration logic here.
2018-12-04 16:08:18 -08:00
Vishnu Ks fc4b6d8f5e auth: Redirect login to realm_redirect page instead of find_accounts. 2018-12-04 09:35:35 -08:00
Vishnu Ks 788b98d041 portico: Add page for redirecting to a realm subdomain. 2018-12-04 09:35:35 -08:00
Eeshan Garg 901100bebf webhooks/jira: Decode topics on our end.
The problem addressed in 006e47198c
resurfaces with topics when a user provides a custom topic in the
webhook URL.
2018-12-03 17:37:36 -08:00
Tim Abbott fa59573a65 mypy: Fix mypy errors in send_email logic.
mypy should be able to figure this out given the xor just above, but
it's not surprising that it doesn't.
2018-12-03 16:30:48 -08:00
Raymond Akornor 92dc3637df send_email: Add support for multiple recipients.
This adds a function that sends provided email to all administrators
of a realm, but in a single email. As a result, send_email now takes
arguments to_user_ids and to_emails instead of to_user_id and
to_email.

We adjust other APIs to match, but note that send_future_email does
not yet support the multiple recipients model for good reasons.

Tweaked by tabbott to modify `manage.py deliver_email` to handle
backwards-compatibily for any ScheduledEmail objects already in the
database.

Fixes #10896.
2018-12-03 15:12:11 -08:00
Tim Abbott 0fddf9a610 docs: Fix 500s with super long fake /help/ article URLs.
While this would never happen for a real article, this prevents a 500
in this case for a situation which is definitely user error and should
be a 40x (in this case, 404).

As part of this, we refactor the main view code to do validation in a
single code path, since the semi-duplicated-in-3-places logic was
getting pretty buggy.
2018-12-03 14:43:45 -08:00
Tim Abbott c740d1b093 clubhouse: Fix typoed include statement. 2018-12-03 14:31:44 -08:00
Tim Abbott 4c1e80032a push notifications: Fix unnecessary warning in development.
This prevents the warning about push notifications not being
registered for from being printed in development environment startup
by default.  In development, that's the expected state, and we don't
need to spam up the output with that notice.
2018-12-03 12:03:56 -08:00
Tim Abbott d96624490e bulk_add_subscriptions: Fix confusing access to user_profile.realm.
Previously, we frequently accessed user_profile.realm from outside the
loops that interact with UserProfile objects.  This variable reuse
outside the loop could be confusing and should be a style/lint
violation.

While in this case, the behavior was correct (in that all users in the
loops were within the same realm), extracting a separate `realm`
variable significantly clarifies what's going on here.
2018-12-03 10:38:10 -08:00
Rishi Gupta 22d8de73e9 webhooks/stripe: Change update_string to provide updated values. 2018-12-03 09:24:52 -08:00
Tim Abbott 48a3975ec0 import: Avoid unnecessary forks when downloading attachments.
The previous implementation used run_parallel incorrectly, passing it
a set of very small jobs (each was to download a single file), which
meant that we'd end up forking once for every file to download.

This correct implementation sends each of N threads 1/N of the files
to download, which is more consistent with the goal of distributing
the download work between N threads.
2018-12-02 13:50:27 -08:00
Rishi Gupta 36f54e689b integrations/stripe: Fix bug in charge.succeeded.
There are (at least) two types of objects that could be sent with a
charge.succeeded event, a Charge (e.g. for credit cards) or a Payment (if
they pay by ACH). We were handling the first but not the second.

This commit also updates the fixture for the existing charge.succeeded event
to the latest API version.
2018-12-02 00:25:52 -08:00
Tim Abbott 5696ad5490 sessions: Suppress coverage error until we can debug. 2018-12-01 10:08:57 -08:00
Tim Abbott 18e2118fc8 Revert "sessions: Stop using the cached_db sessions engine."
This reverts commit 2fa77d9d54.

Further investigation has determined that this did not fix the
password-reset problem described in the previous commit message;
meanwhile, it causes other problems.  We still need to track down the
root cause of the original password-reset bug.
2018-11-30 15:54:32 -08:00
Tim Abbott d0f71881f4 docs: Add detailed documentation on the process for sending messages.
This has long been something missing from our suite of documentation.
2018-11-29 16:25:35 -08:00
Vishnu Ks f3033207b9 emails: Cover all LDAP cases when emailing login details.
This provides a nice user experience for folks where we do know what
their LDAP credentials are.

Though we need to fix #10917 before the content in the email with be
correct.
2018-11-29 12:14:22 -08:00
Tim Abbott 7a2d93ed6e show_admins: Add assertion to make mypy happy. 2018-11-29 11:51:39 -08:00
Tim Abbott 8a60ab7314 show_admins: Rewrite to use management library.
This makes this command more standardized, and helps avoid future bugs
like the one fixed in the last commit.
2018-11-29 11:40:17 -08:00
Tim Abbott d828eba2f6 show_admins: Fix buggy realm parsing. 2018-11-29 11:40:17 -08:00
Rohitt Vashishtha c4e50a34d3 bugdown: Refactor get_user to get_user_by_name.
Also adds a warning against the use of this function.
2018-11-29 10:19:08 -08:00
Hemanth V. Alluri 6b1d724f5c zerver: Add bugdown rendering for text custom profile fields.
This is the first step of letting users use Zulip markdown in their
SHORT_TEXT and LONG_TEXT custom profile fields, so that they can
include emphasis, links, etc.

This doesn't include any frontend logic yet, however.
2018-11-29 09:47:32 -08:00
Anders Kaseorg 1d15d72775 zerver/lib/export.py: Avoid shelling out for cp, rm, ln.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-11-28 17:28:17 -08:00
Eeshan Garg 5ef86b6d22 webhooks/stripe: Handle customer.discount events properly.
Recent changes merged in #10877 didn't handle these events
correctly. The linkified_id function breaks for the `discount`
object in the JSON payload. A cursory glance at Stripe's docs
tells me that since a discount is associated with a customer
or a coupon, it makes sense for a `discount` object to not have
an ID that can necessarily be linked to. So, we can just link
to the associated coupon instead.
2018-11-28 14:16:33 -08:00
Rohitt Vashishtha 681368b937 bugdown: Refactor get_possible_mentions_info and related functions.
This commit changes the return type of get_possible_mentions_info  to a
list instead of a dict, thus disposing off the hacky logic of storing
users with duplicate full names with name|id keys that made the code
obfuscated.

The other functions continue to use the dicts as before, however, there
are minor variable changes where needed in accordance with the updated
definition of get_possible_mentions_info.
2018-11-28 14:07:52 -08:00
Tim Abbott e4946dd182 bugdown: Rename full_names to mention_texts.
This is another straight variable rename, which will help clarify the
upcoming commits.
2018-11-28 14:07:23 -08:00
Rohitt Vashishtha ccdf893af7 bugdown: Rename get_full_name_info to get_possible_mentions_info. 2018-11-28 14:04:50 -08:00
Rishi Gupta dd0126ff1b settings: Improve error message when deactivating the last user.
This PR was originally started by Rishi Gupta (see #10383).
2018-11-27 12:49:43 -08:00
Eeshan Garg 7fb674cc58 messages: Add Addressee.for_user_ids().
This commit is a part of our efforts surrounding #9474.
2018-11-27 12:09:41 -08:00
Eeshan Garg 91b90460ee lib/actions: Add helper recipient_for_user_ids().
This function is equivalent to recipient_for_emails, but fetches
user_profiles by IDs, not by emails.

This commit is a part of our efforts surrounding #9474, but is
more primarily geared towards adding support for sending typing
notifications by user IDs.
2018-11-27 11:59:11 -08:00
Eeshan Garg 8ffc437e70 addressee: Raise JsonableError in get_user_profiles().
Previously, get_user_profiles() was split into two functions:

* user_profiles_from_unvalidated_emails, which raised a
  ValidationError upon encountering a non-existent user email.
* get_user_profiles, which caught the ValidationError raised
  by user_profiles_from_unvalidated_emails and raised a
  JsonableError instead.

According to Steve Howell, this complexity is partly a relic
of past refactoring and is unnecessarily heavy. It is better to
just raise JsonableError directly.
2018-11-27 11:59:11 -08:00
Eeshan Garg 68a06abb79 actions: Stop using Addressee's helpers in recipient_for_emails.
recipient_for_emails is used by our typing notifications code.
user_profiles_from_unvalidated_emails is used by our typing
notifications code *and* for sending messages.
user_profiles_from_unvalidated_emails is a part of a larger
framework used by Addressee to validate recipient emails when sending
messages and will eventually need to be removed as we move forward
with #9474. So it makes sense to just inline this function within
recipient_for_emails so that we don't break our typing notifications
code in the future.

This commit is a part of our efforts surrounding #9474.
2018-11-27 11:59:11 -08:00
Tim Abbott adf27aae4c python: Remove now-unnecessary str_utils library.
This library was absolutely essential as part of our Python 2->3
migration process, but all of its calls should be either no-ops or
encode/decode operations.

Note also that the library has been wrong since the incorrect
refactoring in 1f9244e060.

Fixes #10807.
2018-11-27 11:57:54 -08:00
Tim Abbott 6a9e6503b7 python: Remove unnecessary NonBinaryStr TypeVar.
This was a potentially useful value (later superceded by use of Text)
when first introduced, but now that we're on Python 3, it should just
be str.
2018-11-27 11:44:09 -08:00
Tim Abbott c679920c01 python: Fix unnecessary uses of str_utils library. 2018-11-27 11:44:09 -08:00
Tim Abbott e06668c7e8 queue_processors: Fix misleading copied comment.
This comment was clearly copied from the previous processor.
2018-11-27 11:44:09 -08:00
Raymond Akornor d00b889402 auth: Add an organization reactivation flow with admin confirmation.
This adds a web flow and management command for reactivating a Zulip
organization, with confirmation from one of the organization
administrators.

Further work is needed to make the emails nicer (ideally, we'd send
one email with all the admins on the `To` line, but the `send_email`
library doesn't support that).

Fixes #10783.

With significant tweaks to the email text by tabbott.
2018-11-27 10:58:34 -08:00
Tim Abbott 10e8e2acac push notifications: Tweak text for push notification message.
The previous content made it sound like we were actually sending a
push notification, which could be confusing/alarming in some cases
(see e.g. 9c224ccdd3).  Instead, we make
clear that we're sending it to all clients (which one might correctly
suspect is vacuous in the development environment).
2018-11-27 09:53:44 -08:00
Tim Abbott 7b930124d9 push notifications: Add a logger (default-off in tests).
This should suppress some spammy logging output about push
notifications that we were seeing in a large number of unit tests.
2018-11-27 09:45:45 -08:00
Tim Abbott 38a6003472 push notifications: Improve logging for missing configuration.
While it could make sense to print these logging statements at WARN
level on server startup, it doesn't make sense to do so on every
message (though it perhaps did make sense to do so before more recent
changes added good ways to discover you forgot to configure push
notifications).

Instead, we now just do a WARN log on queue processor startup, and
then at DEBUG level for individual messages.

Fixes #10894.
2018-11-27 09:37:57 -08:00
Tim Abbott beb80219a8 push notifications: Fix a comment typo. 2018-11-27 09:18:35 -08:00
Tim Abbott 3bb0956046 upload: Fix ensure_medium_avatar_image for S3 backend.
Previously, it tried to interact with the wrong path for the original
image.
2018-11-26 20:20:27 -08:00
Tim Abbott 00826486bd hipchat: Fix typo in logging output. 2018-11-26 16:44:31 -08:00
Steve Howell 38f81d5d20 hipchat: Skip public stream subs in slim mode. 2018-11-26 16:37:30 -08:00
Steve Howell c2e9f5eb0a hipchat: Limit messages in slim mode.
For messages with strange senders, we don't import
messages.  Basically, we only import a message if
it has sender with an id that maps to a non-deleted
user.
2018-11-26 16:37:30 -08:00
Steve Howell 3a7788217e hipchat: Skip really long messages. 2018-11-26 16:37:30 -08:00
Steve Howell e57a932692 hipchat: Fix avatars.
This code was not reading any avatars because
it was not referencing 'User' to get to the avatar,
and it was not re-mapping user ids for some reason.
2018-11-26 16:37:30 -08:00