Commit Graph

9268 Commits

Author SHA1 Message Date
Shoumorup e8ba08367b requirements: Upgrade pyflakes.
Pyflakes has been upgraded from 2.0.0 to 2.1.0 and
a few new linter errors have been fixed.

Fixes #11397.
2019-02-01 07:26:13 -08:00
Tim Abbott 1bcd7dfbad remote_server: Return the result dictionary on success. 2019-01-31 17:39:41 -08:00
Tim Abbott 2d11e163dd push_notifications: Move "push" part of URLs to callers.
This will make it possible for us to use this library for endpoints
not directly related to push notifications.
2019-01-31 15:22:00 -08:00
Rishi Gupta 4fab08172b billing: Remove deprecated billing fields from Realm. 2019-01-31 15:12:43 -08:00
Rishi Gupta 8f7e38c559 billing: Remove deprecated billing tables and fields from corporate/models. 2019-01-31 15:12:43 -08:00
Rishi Gupta 55179f913c billing: Remove references to Customer.has_billing_relationship. 2019-01-31 15:12:43 -08:00
Tim Abbott 88fae0b6a9 remote_server: Extract remote_server.py library.
This moves the network request code for connecting to the push
notification bouncer service into its own module.
2019-01-31 15:08:46 -08:00
Eeshan Garg f4221f2655 internal_prep_message: Don't call ensure_stream for stream IDs.
Ever since we implemented support for stream IDs in Addressee,
Addressee.stream_name() can now return None. This commit ensures
that _internal_prep_message only calls ensure_stream when
Addressee.stream_name() is not None.
2019-01-31 12:55:10 -08:00
Eeshan Garg 0420ecadeb addressee: Use stream ID for sender's default sending stream. 2019-01-31 12:43:38 -08:00
Eeshan Garg 998437c123 check_message: Support sending stream messages by ID.
This commit also contains the following auxiliary changes:
* Adds a custom exception, StreamWithIDDoesNotExist for when
  a stream with a given ID does not exist because the error
  message returned by StreamDoesNotExist only makes with stream
  names, not IDs.
* Adds a new helper, get_stream_by_id_in_realm, which is similar
  to get_user_profile_by_id_in_realm (introduced in #10391).
* Adds a helper, validate_stream_id_with_pm_notification, which
  returns the Stream object associated with a given ID and also
  handles PM notifications to the bot owner if the message was
  sent by a bot and if the stream does not exist or has no
  subscribers.
* Modifies the message sent by send_pm_if_empty_stream to
  accommodate stream IDs.

Note that all of the above changes are required before check_message
can be modified to support stream IDs.
2019-01-31 12:43:38 -08:00
Eeshan Garg b8221555d2 addressee: Add support for stream IDs to Addressee.legacy_build(). 2019-01-31 12:43:38 -08:00
Tim Abbott 9b25f8789f hipchat: Fix handling of user IDs in Stride import.
We've had this code oscillated a few times; the original comparison
was added as part of Stride import but broke HipChat import.
c34a8f2e69 fixed HipChat import but
regressed Stride.

This change fixes this for both HipChat + Stride.
2019-01-31 12:40:05 -08:00
Pragati Agrawal e1772b3b8f tools: Upgrade Pycodestyle and fix new linter errors.
Here, we are upgrading pycodestyle version from 2.4.0 to 2.5.0.

Fixes: #11396.
2019-01-31 12:21:41 -08:00
Vishnu Ks 0d0007742f requirements: Upgrade pika from 0.12.0 to 0.13.0.
The important changes to pika for us are based on this PR of ours:
https://github.com/pika/pika/pull/1129

Fixes #11394.
2019-01-31 10:04:07 -08:00
Harshit Bansal 837483b026 tests: Don't fail silently on 500 on logins. 2019-01-29 16:01:30 -08:00
Harshit Bansal 6777b94d41 lib: Extract `validate_user_custom_profile_field()`. 2019-01-29 16:01:30 -08:00
Harshit Bansal 9bac192241 dev_ldap: Add custom profile data. 2019-01-29 16:01:30 -08:00
Harshit Bansal 88c1d868d2 dev_ldap: Fix password in mode 'c' to be equal to username. 2019-01-29 16:01:30 -08:00
Harshit Bansal dccfadc1a5 minor: Fix some typing errors. 2019-01-29 16:01:30 -08:00
Rishi Gupta 78d98168a4 portico: Add stride import to /features and /help. 2019-01-29 13:48:39 -08:00
Steve Howell c2fcfc087a bugdown: Include message id in exceptions. 2019-01-29 12:49:56 -08:00
Steve Howell 643d4cacd3 bugdown tests: Pass message stub to bugdown_convert().
This makes the calling sequence slightly more
realistic (without any additional overhead).
2019-01-29 12:49:56 -08:00
Tim Abbott 8d404dbcc6 confirmation: Fix "continue to registration" full name handling.
Apparently, the "continue to registration" flow used a subtly invalid
way of encoding the full name.  We put in the query part of the action
URL of the HTML form, but apparently HTML forms with a `GET` type will
ignore the query part (replacing it with any input values), which
makes sense but doesn't do what we want here.  There are a few sane
ways to fix it, but given that the encoding logic we had before for
including the name in the URL was ugly, I'm pretty happy with just
adding a hidden input to the form for the name.
2019-01-29 12:39:40 -08:00
Tim Abbott 9dc4b04e52 auth: Migrate Google authentication off deprecated name API.
As part of Google+ being removed, they've eliminated support for the
/plus/v1/people/me endpoint.  Replace it with the very similar
/oauth2/v3/userinfo endpoint.
2019-01-29 12:39:40 -08:00
Tim Abbott aea93f9826 api: Document support for setting stream description on creation.
This makes clear how to set the stream description when making a new
stream.

Fixes #11372.
2019-01-29 11:28:55 -08:00
Daniel Bradburn 1e65cdd072 emoji: Only resize custom emoji that need it.
This additional logic to prevent resizing is certain circumstances
(file size, dimensions) is necessary because the pillow gif handling
code seems to be rather flaky with regards to handling gif color
palletes, causing broken gifs after resizing.  The workaround is to
only resize when absolutely necessary (e.g. because the file is larger
than 128x128 or 128KB).

Fixes #10351.
2019-01-29 10:33:50 -08:00
Rohitt Vashishtha a2751d90c2 widgets: Remove ALLOW_SUB_MESSAGES flag and enable widgets by default. 2019-01-29 09:34:14 -08:00
Rohitt Vashishtha c176891c2e poll-widget: Refactor comment to option.
We had initially designed the poll widget like a blog
post with comments beneath it but it makes more sense
to think of it as just a simple poll with options.
2019-01-29 09:34:14 -08:00
Rohitt Vashishtha 5641afc6e2 poll-widget: Add syntax for adding options when creating poll.
We add a new syntax which converts the messages like the following:

```
/poll Who do you support?

Nadal
- Djokovic
```

to a poll with the two names as options. The list syntax is optional
since anyone making a poll is likely to want to create a list anyway.
2019-01-29 09:34:14 -08:00
Tim Abbott 0181086d0f streams: Improve validation for arguments to stream creation.
This doesn't have any security impact, since we overwrote any other
fields in any case, and also this step happens before the security
part of input validation for stream creation.  But this does improve
error messages if one tries to specify other arguments, and also makes
more clear that the `description` argument is supported here.
2019-01-28 22:02:16 -08:00
Akshay Gupta 88ed75d06d travis_webhooks: Update the list of bad statuses.
Fixes #11259.
2019-01-28 21:37:18 -08:00
Adarsh Patel 9b740df277 api_docs: Specify max width for long code examples.
Fixes #10673.
2019-01-28 22:39:54 -03:30
Rishi Gupta 7c11fe819a billing: Update license ledger when users are added and removed. 2019-01-28 16:04:11 -08:00
Wyatt Hoodes 8eac361fb5 docs: Refactor BS work with use of cache_with_key.
Refactor the potentially expensive work done by Beautiful Soup into a
function that is called by the alter_content function, so that we can
cache the result.  Saves a significant portion of the runtime of
loading of all of our /help/ and /api/ documentation pages (e.g. 12ms
for /api).

Fixes #11088.

Tweaked by tabbott to use the URL path as the cache key, clean up
argument structure, and use a clearer name for the function.
2019-01-28 15:21:52 -08:00
Matthew Wegner 370cf1a2cb import: Normalize Slackbot String Comparison.
In very old Slack workspaces, slackbot can appear as "Slackbot", and
the import script only checks for "slackbot" (case sensitive).  This
breaks the import--it throws the assert that immediately follows the
test.  I don't know how common this is, but it definitely affected our
import.

The simple fix is to compare against a lowercased-version of the
user's full name.
2019-01-28 14:59:41 -08:00
Steve Howell f2b26b9bcc integration tests: Add check for new messages.
This commit should make it a bit easier to debug
integrations that silently don't send any messages.
2019-01-28 14:56:42 -08:00
Rohitt Vashishtha ff90c0101c bugdown: Process word boundaries properly in realm_filters.
Earlier, our realm filters didn't render for languages that do not
use spaces (eg: Japanese) since we used to check for the presence
of an actual space character. This commit replaces that logic with
a complex scheme to detect word boundaries.

Also, we convert the RealmFilterPattern to subclass InlineProcessor
and make use of the new no-op feature in py-markdown 3.0.1 where we
can tell py-markdown that our pattern didn't find a match despite
the initial regex getting matched.

Fixes #9883.
2019-01-28 14:48:15 -08:00
Steve Howell ad071ced47 bugdown: Avoid recomputing the stream-link regex. 2019-01-28 13:12:37 -08:00
Rohitt Vashishtha 2dc447d707 bugdown: List py-markdown 3.0.1 features that we do not use.
Tweaked by tabbott to extend the documentation.
2019-01-28 13:12:37 -08:00
Rohitt Vashishtha 434094e599 bugdown: Restructure Bugdown to extend Markdown from being an extension.
Since we are building our parser from scratch now:

1. We have control over which proccessor goes at what priority number.
   Thus, we have also shifted the deprecated `.add()` calls to use the
   new `.register()` calls with explicit priorities, but maintaining
   the original order that the old method generated.

2. We do not have to remove the processors added by py-markdown that
   we do not use in Zulip; we explicitly add only the processors we
   do require.

3. We can cluster the building of each type of parser in one place,
   and in the order they need to be so that when we register them,
   there is no need to sort the list. This also makes for a huge
   improvement in the readability of the code, as all the components
   of each type are registered in the same function.

These are significant performance improvements, because we save on
calls to `str.startswith` in `.add()`, all the resources taken to
generate the default to-be-removed processors and the time taken to
sort the list of processors.

Following are the profiling results for the changes made. Here, we
build 10 engines one after the other and note the time taken to build
each of them. 1st pass represents the state after this commit and 2nd
pass represent the state after some regex modifications in the commits
that follow by Steve Howell. All times are in microseconds.

| nth Engine | Old Time | 1st Pass | 2nd Pass |
| ---------- | -------- | -------- | -------- |
|          1 |  92117.0 |  81775.0 |  76710.0 |
|          2 |   1254.0 |    558.0 |    341.0 |
|          3 |   1170.0 |    472.0 |    305.0 |
|          4 |   1155.0 |    519.0 |    301.0 |
|          5 |   1170.0 |    546.0 |    326.0 |
|          6 |   1271.0 |    609.0 |    416.0 |
|          7 |   1125.0 |    459.0 |    299.0 |
|          8 |   1146.0 |    476.0 |    390.0 |
|          9 |   1274.0 |    446.0 |    301.0 |
|         10 |   1135.0 |    451.0 |    297.0 |
2019-01-28 13:12:37 -08:00
Rohitt Vashishtha 9f2c52c86e bugdown: Rename variables regex to REGEX for importing regex module. 2019-01-28 12:00:58 -08:00
Steve Howell 3b7d899532 bugdown: Use CompiledPattern in AtomicLinkPattern.
We avoid re-computing the regex string here, and we
also avoid re-compiling the regex itself.

I decided to put the "one_time" decorator in the
bugdown file itself, just to reduce friction in
folks reading the "buyer beware" comments.

Unfortunately, we can't use this for the
get_web_link_regex() function due to testing concerns,
so that continues to do an inelegant cache-with-global-var
scheme.
2019-01-28 11:58:47 -08:00
Steve Howell eea711a805 bugdown: Flatten get_web_link_regex().
We use early-exit to flatten the code.

I also tweaked the comments a bit based on some recent
profile findings.  (e.g. reading the file isn't actually
a big bottleneck, it's more the regex itself)
2019-01-28 11:58:46 -08:00
Steve Howell 852756aeb3 bugdown: Eliminate LinkPattern class.
The only code that used LinkPattern was AtomicLinkPattern.

We just move the helper method into AtomicLinkPattern.
2019-01-28 11:58:05 -08:00
Steve Howell 77446a710c bugdown: Optimize CompiledPattern.
We don't need the hacky step of passing in a blank
regex to the superclass's __init__ function.

All we need to do is assign `md` to `self.md`.
2019-01-28 11:57:28 -08:00
Steve Howell 025df33d7a bugdown: Rename VerbosePattern to CompiledPattern.
This class only requires a compiled regex--it's up
to the callers how they compile it (verbose or
otherwise).
2019-01-28 11:56:01 -08:00
Eeshan Garg 681f07f98f check_message: Refactor stream validation into separate function. 2019-01-28 11:48:17 -08:00
Eeshan Garg 4cb9a472e6 addressee: Add helper to check for missing or empty topic. 2019-01-28 11:42:26 -08:00
kunal-mohta b89769420a stream_color: Extend subscribe API to support color attribute.
This fixes an annoying bug where clicking to subscribe to a stream
would change the color shown in the "manage streams" UI immediately
after you click.

Fixes #11072.
2019-01-25 16:44:59 -08:00
Pragati Agrawal 1e811b42ec org settings: Add realm level setting for missed message content in email.
This adds a setting under "Notification" section of
"Organization settings" tab, which enables Organization administrator to
control whether the missed message emails include the message content or
not.

Fixes: #11123.
2019-01-25 14:34:10 -08:00