Commit Graph

28505 Commits

Author SHA1 Message Date
Steve Howell 76deb30312 preview: Hash cache keys for preview urls.
We don't want really long urls to lead to truncated
keys, or we could theoretically have two different
urls get mixed up previews.

Also, this suppresses warnings about exceeding the
250 char limit.

Finally, this gives the key a proper prefix.
2018-10-14 09:28:57 -07:00
Steve Howell d933779477 hipchat import: Support PrivateUserMessage data.
We now import PM data from HipChat.
2018-10-13 16:47:44 -07:00
Steve Howell f0c3ee0a2e hipchat import: Write smaller message files.
We now write new message files for each new input
file + message type we process.  This helps the
importer not run out of memory later.
2018-10-13 16:47:44 -07:00
Steve Howell 75fc5d41c9 hipchat import: Refactor write_message_data.
The goal here is to make it easier to handle other
message types by moving the key-specific stuff
to the top of the file.
2018-10-13 16:47:44 -07:00
Steve Howell cc55eb8154 hipchat import: Only process UserMessage rows for now. 2018-10-13 16:47:44 -07:00
Steve Howell 3baac7ddf3 hipchat import: Handle missing emails for guest users. 2018-10-13 16:47:44 -07:00
Steve Howell 8accc60ca7 import_util: Support multiple message ids for attachments. 2018-10-13 16:47:44 -07:00
Steve Howell 23d7b3d2cc import: De-dup create_converted_data_files helper. 2018-10-13 16:47:41 -07:00
Steve Howell 91905bd66a import: Add sequencer library.
This avoids some tedious code related to making ids
in conversion programs.
2018-10-13 16:47:39 -07:00
Steve Howell 85f1910f93 minor: Add link to hipchat spec to code. 2018-10-13 16:43:28 -07:00
Steve Howell 493aae2958 imports: Make loading UserMessage faster and more robust.
We use UserMessageLite to avoid Django overhead, and we
do updates in chunks of 10000.  (The export may be broken
into several files already, but a reasonable chunking at
import time is good defense against running out of memory.)
2018-10-13 16:43:28 -07:00
Rohitt Vashishtha 0d8ce4201c composebox_typeahead.js: Use people.get_mention_syntax. 2018-10-13 16:42:50 -07:00
Rohitt Vashishtha f857111920 popovers.js: Use people.get_mention_syntax. 2018-10-13 16:42:50 -07:00
Rohitt Vashishtha 11e4011d5f transmit.js: Use people.get_mention_syntax. 2018-10-13 16:42:50 -07:00
Rohitt Vashishtha 38cddf7d54 compose_actions: Use people.get_mention_syntax in reply_with_mention. 2018-10-13 16:42:50 -07:00
Rohitt Vashishtha b18f9def06 people.js: Add get_mention_syntax to conditionally get @user|id syntax.
This is intended to replace all function calls for generating mention
syntax for a target user.
2018-10-13 16:42:34 -07:00
Tim Abbott 608173657d upgrade-zulip: Clean up legacy google emoji icons.
Because we renamed the "google" iconset to be the modern Google set,
not what is now called the "googleblob" icon set, we need to make sure
that our usually correct policy of not overwriting image files under
`prod-static/` doesn't apply to files potentially being copied in for
the emoji images.

We fix this by just deleting the `images-google-64` directory on
upgrade if it contains the googleblob version of the "hotdog" emoji.

Fixes #10038.
2018-10-12 16:38:49 -07:00
Tim Abbott 3c2f49edaa message_flags: Fix JS exception when handling starred messages.
Previously, if one starred a message that wasn't in the browser's
locally cached history, it'd throw a JavaScript exception.
2018-10-12 13:52:30 -07:00
Steve Howell 27ef6a9991 compose pill: Add has_unconverted_data() helper. 2018-10-12 11:20:30 -07:00
Steve Howell 3225f81a92 compose pills: Add get_user_ids_string().
This helper will be useful for typing indicators.
2018-10-12 11:20:30 -07:00
Steve Howell 3b9a5afcee input_pill: Add is_pending() helper. 2018-10-12 11:20:30 -07:00
Tim Abbott 68ab71eb8b push: Fix exceptions when removing push notifications.
Now that we allow multiple users to have registered the same token, we
need to configure calls to unregister tokens to only query the
targeted user_id.

We conveniently were already passing the `user_id` into the push
notification bouncer for the remove API, so no migration for older
Zulip servers is required.
2018-10-12 11:19:23 -07:00
Steve Howell 3545bc1893 settings: Confirm before deleting user groups.
Fixes #10498.
2018-10-12 10:38:56 -07:00
Steve Howell 19ca7145fc settings: Extract delete_user_group().
This preps us to make delete_user_group() be
a callback in the next commit.
2018-10-12 10:38:54 -07:00
Steve Howell b7f764aa29 settings: Add confirm_dialog module.
This module makes it really easy to create are-you-sure
dialogs for dangerous operations.

Basically it's one function with five parameters.  You
give three chunks of HTML, a callback function, and
a parent container.

The first use of this will be in settings_user_groups,
coming up in a couple commits.
2018-10-12 10:37:06 -07:00
Akash Nimare f60e6d7c20 desktop: Update app to v2.3.82. 2018-10-12 10:32:58 -07:00
Steve Howell 0f7628280f narrow: Handle spurious emails in pm-with searches.
If cordelia searches on pm-with:iago@zulip.com,cordelia@zulip.com,
we now properly treat that the same way as pm-with:iago@zulip.com.

Before this fix, the query would initially go through the
huddle code path.  The symptom wasn't completely obvious, as
eventually a deeper function would return a recipient id
corresponding to a single PM with @iago@zulip.com, but we would
only get messages where iago was the recipient, and not any
messages where he was the sender to cordelia.

I put the helper function for this in zerver/lib/addressee, which
is somewhat speculative.  Eventually, we'll want pm-with queries
to allow for user ids, and I imagine there will be some shared
logic with other Addressee code in terms of how we handle these
strings.  The way we deal with lists of emails/users for various
endpoints is kind of haphazard in the current code, although
granted it's mostly just repeating the same simple patterns.  It
would be nice for some of this code to converge a bit.  This
affects new messages, typing indicators, search filters, etc.,
and some endpoints have strange legacy stuff like supporting
JSON-encoded lists, so it's not trivial to clean this up.

Tweaked by tabbott to add some additional tests.
2018-10-12 10:18:30 -07:00
Steve Howell 9f2aad55b5 hipchat import: Handle users without avatars. 2018-10-12 07:03:25 -04:00
Tim Abbott cc066032f6 auth: Add invalid default value for auth_backend_name.
This makes mypy happy with the new logic we just added.
2018-10-11 17:35:07 -07:00
Tim Abbott 792b08c3b8 auth: Remove unused CSS/HTML for social auth buttons.
The margin-left CSS was incorrect and overridden anyway, and after
removing that, login-github-button no longer appeared in the project.
2018-10-11 17:18:04 -07:00
Tim Abbott 20a13b42e6 auth: Automatically maintain list of OAuth backends.
Here, we take advantage of the useful issubclass function to determine
which of our social backends are based on OAuth2.
2018-10-11 17:11:04 -07:00
Tim Abbott 1244a5077e auth: Automate social auth backends in AUTH_BACKEND_NAME_MAP.
This saves a line of manual code every time we add a new social auth
backend.
2018-10-11 17:11:03 -07:00
Tim Abbott fd4c23e12d login: Clean up CSS/HTML for google/github login buttons.
This removes some unnecessary code duplication in the CSS classes for
Google and GitHub authentication social auth buttons.

This will, in turn, help us avoid extra work every time we add a new
authentication backend.
2018-10-11 16:52:41 -07:00
Tim Abbott b51633efb7 auth: Allow '-' characters in social auth login urls.
This is important for backends like azuread-oauth2.
2018-10-11 16:44:33 -07:00
Steve Howell 51bd36e448 tests: Add coverage to get_service_interface_class(). 2018-10-11 16:12:07 -07:00
Steve Howell 8379aeee15 outgoing bots: Fix header for generic servers.
For our bots that use GenericOutgoingWebhookService
(which are basically Zulip style bots), we now
include a "content-type" header of "application/json".

We accomplish this by having the service classes
implement their own custom method called
`send_data_to_server`. For the Slack-related
code, we just extracted code from `do_rest_call`,
and then for the Zulip-related code, we added
a `headers` parameter.
2018-10-11 16:12:07 -07:00
Steve Howell 8226e13e9c bot tests: Replace use of MockServiceHandler.
We'll just use a real class here, since the service
handlers are pretty lightweight and just munge data.
2018-10-11 16:12:07 -07:00
Steve Howell 8f74d99b6c Remove stubs in OutgoingWebhookServiceInterface.
If we omit methods in subclasses, they're likely to
be caught by linters or unit tests, and even if they
aren't, raising NotImplementedError doesn't actually
prevent user problems.

I've been fighting these in refactoring, and it's
just been a bunch of busy work, plus comments are
highly likely to bitrot.
2018-10-11 16:12:07 -07:00
Steve Howell 31597cf33e Remove timeout parameter in do_rest_call().
Nobody was setting it.
2018-10-11 16:12:07 -07:00
Steve Howell 69ee84bb14 refactor: Extract build_bot_request().
This fixes a couple things:

    * process_event() is a pretty vague name
    * returning tuples should generally be avoided
    * we were producing the same REST parameters in both
      subclasses
    * relative_url_path was always blank
    * request_kwargs was always empty

Now process_event() is called build_bot_request(),
and it only returns request data,
not a tuple of `rest_operation` and `request_data`.

By no longer returning `rest_operation`, there are
fewer moving parts.  We just have `do_rest_call` make
a POST call.
2018-10-11 16:12:07 -07:00
Steve Howell 16eff75e49 refactor: Simplify how we use base_url.
Before this change, we instantiated base_url into a superclass
of subclasses that returned base_url into a dictionary that
gets returned to our caller.

Now we just pull base_url out of service when we need to make
the REST call.
2018-10-11 16:12:07 -07:00
Steve Howell b89a94f730 Improve errors when we can't connect to a bot server.
We don't overwhelm people with error info when bots
fail to connect or time out.
2018-10-11 16:12:07 -07:00
Steve Howell 3790c469e9 outgoing bots: Report JSON errors to users.
We should arguably report these to bot owners
as well, but this is at least an improvement
over having the server crash.
2018-10-11 16:12:07 -07:00
Steve Howell df4b665658 refactor: Parse JSON from bots in one place.
We move the JSON parsing step into the
higher level function: process_success_response().

In the unlikely event that we'll start integrating
with a solution that doesn't use JSON, we can deal
with that, and for now doing the parsing in one
place will help us make error reporting more
consistent.

In a subsequent commit we'll introduce better
error handling for malformed JSON.
2018-10-11 16:12:07 -07:00
Steve Howell 395d74c08a api docs: Clean up outgoing webhooks section.
I rewrote the section explaining what the
endpoint sends back to the server.  This fixes
a few typos, emphasizes the normal case, and
starts to favor "content" as the key for
content.
2018-10-11 16:12:07 -07:00
Steve Howell 229dd5d861 outgoing webhooks: Get rid of "Success!" prefix.
The earlier code here, if it got a payload with
"response_string" as a key, would prefix the
corresponding value with "Success!".  We just
want the bot to set its own content.

The code is reorganized here so that process_success()
always produces a value keyed by "content" from
incoming data, and then process_success_response()
doesn't do any fancy munging of the data.
2018-10-11 16:12:07 -07:00
Tim Abbott 0a751567a3 upload: Fix missing mypy return type annotation. 2018-10-11 16:11:20 -07:00
Joshua Pan 971cb18cb3 user_settings: Compare new stripped email with old email.
We weren't comparing the newly stripped email with the current
old email, thus adding spaces around an email would result in
an error.
2018-10-11 15:55:32 -07:00
Tim Abbott 8cf104b643 avatar: Allow API authentication for /avatar/ routes.
This makes it feasibly for the mobile apps to correctly render user
avatars generated by the `!avatar()` syntax.
2018-10-11 15:52:29 -07:00
Aditya Bansal 3164f1a9a4 avatar: Rename user_profile to avatar_user_profile.
This is a preparatory commit for upcoming changes to move
/avatar/ to be a logged in or API accessible endpoint.
Basically we rename this variable because the new name is more
appropriate in the situation. Also user_profile will be used to
hold the user_profile of person accessing the endpoint in coming up
commit.
2018-10-11 15:50:37 -07:00