Commit Graph

4041 Commits

Author SHA1 Message Date
Pragati Agrawal e1772b3b8f tools: Upgrade Pycodestyle and fix new linter errors.
Here, we are upgrading pycodestyle version from 2.4.0 to 2.5.0.

Fixes: #11396.
2019-01-31 12:21:41 -08:00
Vishnu Ks 0d0007742f requirements: Upgrade pika from 0.12.0 to 0.13.0.
The important changes to pika for us are based on this PR of ours:
https://github.com/pika/pika/pull/1129

Fixes #11394.
2019-01-31 10:04:07 -08:00
Harshit Bansal 837483b026 tests: Don't fail silently on 500 on logins. 2019-01-29 16:01:30 -08:00
Harshit Bansal 6777b94d41 lib: Extract `validate_user_custom_profile_field()`. 2019-01-29 16:01:30 -08:00
Harshit Bansal 9bac192241 dev_ldap: Add custom profile data. 2019-01-29 16:01:30 -08:00
Harshit Bansal 88c1d868d2 dev_ldap: Fix password in mode 'c' to be equal to username. 2019-01-29 16:01:30 -08:00
Harshit Bansal dccfadc1a5 minor: Fix some typing errors. 2019-01-29 16:01:30 -08:00
Rishi Gupta 78d98168a4 portico: Add stride import to /features and /help. 2019-01-29 13:48:39 -08:00
Steve Howell c2fcfc087a bugdown: Include message id in exceptions. 2019-01-29 12:49:56 -08:00
Daniel Bradburn 1e65cdd072 emoji: Only resize custom emoji that need it.
This additional logic to prevent resizing is certain circumstances
(file size, dimensions) is necessary because the pillow gif handling
code seems to be rather flaky with regards to handling gif color
palletes, causing broken gifs after resizing.  The workaround is to
only resize when absolutely necessary (e.g. because the file is larger
than 128x128 or 128KB).

Fixes #10351.
2019-01-29 10:33:50 -08:00
Rohitt Vashishtha a2751d90c2 widgets: Remove ALLOW_SUB_MESSAGES flag and enable widgets by default. 2019-01-29 09:34:14 -08:00
Rohitt Vashishtha c176891c2e poll-widget: Refactor comment to option.
We had initially designed the poll widget like a blog
post with comments beneath it but it makes more sense
to think of it as just a simple poll with options.
2019-01-29 09:34:14 -08:00
Rohitt Vashishtha 5641afc6e2 poll-widget: Add syntax for adding options when creating poll.
We add a new syntax which converts the messages like the following:

```
/poll Who do you support?

Nadal
- Djokovic
```

to a poll with the two names as options. The list syntax is optional
since anyone making a poll is likely to want to create a list anyway.
2019-01-29 09:34:14 -08:00
Adarsh Patel 9b740df277 api_docs: Specify max width for long code examples.
Fixes #10673.
2019-01-28 22:39:54 -03:30
Rishi Gupta 7c11fe819a billing: Update license ledger when users are added and removed. 2019-01-28 16:04:11 -08:00
Wyatt Hoodes 8eac361fb5 docs: Refactor BS work with use of cache_with_key.
Refactor the potentially expensive work done by Beautiful Soup into a
function that is called by the alter_content function, so that we can
cache the result.  Saves a significant portion of the runtime of
loading of all of our /help/ and /api/ documentation pages (e.g. 12ms
for /api).

Fixes #11088.

Tweaked by tabbott to use the URL path as the cache key, clean up
argument structure, and use a clearer name for the function.
2019-01-28 15:21:52 -08:00
Steve Howell f2b26b9bcc integration tests: Add check for new messages.
This commit should make it a bit easier to debug
integrations that silently don't send any messages.
2019-01-28 14:56:42 -08:00
Rohitt Vashishtha ff90c0101c bugdown: Process word boundaries properly in realm_filters.
Earlier, our realm filters didn't render for languages that do not
use spaces (eg: Japanese) since we used to check for the presence
of an actual space character. This commit replaces that logic with
a complex scheme to detect word boundaries.

Also, we convert the RealmFilterPattern to subclass InlineProcessor
and make use of the new no-op feature in py-markdown 3.0.1 where we
can tell py-markdown that our pattern didn't find a match despite
the initial regex getting matched.

Fixes #9883.
2019-01-28 14:48:15 -08:00
Steve Howell ad071ced47 bugdown: Avoid recomputing the stream-link regex. 2019-01-28 13:12:37 -08:00
Rohitt Vashishtha 2dc447d707 bugdown: List py-markdown 3.0.1 features that we do not use.
Tweaked by tabbott to extend the documentation.
2019-01-28 13:12:37 -08:00
Rohitt Vashishtha 434094e599 bugdown: Restructure Bugdown to extend Markdown from being an extension.
Since we are building our parser from scratch now:

1. We have control over which proccessor goes at what priority number.
   Thus, we have also shifted the deprecated `.add()` calls to use the
   new `.register()` calls with explicit priorities, but maintaining
   the original order that the old method generated.

2. We do not have to remove the processors added by py-markdown that
   we do not use in Zulip; we explicitly add only the processors we
   do require.

3. We can cluster the building of each type of parser in one place,
   and in the order they need to be so that when we register them,
   there is no need to sort the list. This also makes for a huge
   improvement in the readability of the code, as all the components
   of each type are registered in the same function.

These are significant performance improvements, because we save on
calls to `str.startswith` in `.add()`, all the resources taken to
generate the default to-be-removed processors and the time taken to
sort the list of processors.

Following are the profiling results for the changes made. Here, we
build 10 engines one after the other and note the time taken to build
each of them. 1st pass represents the state after this commit and 2nd
pass represent the state after some regex modifications in the commits
that follow by Steve Howell. All times are in microseconds.

| nth Engine | Old Time | 1st Pass | 2nd Pass |
| ---------- | -------- | -------- | -------- |
|          1 |  92117.0 |  81775.0 |  76710.0 |
|          2 |   1254.0 |    558.0 |    341.0 |
|          3 |   1170.0 |    472.0 |    305.0 |
|          4 |   1155.0 |    519.0 |    301.0 |
|          5 |   1170.0 |    546.0 |    326.0 |
|          6 |   1271.0 |    609.0 |    416.0 |
|          7 |   1125.0 |    459.0 |    299.0 |
|          8 |   1146.0 |    476.0 |    390.0 |
|          9 |   1274.0 |    446.0 |    301.0 |
|         10 |   1135.0 |    451.0 |    297.0 |
2019-01-28 13:12:37 -08:00
Rohitt Vashishtha 9f2c52c86e bugdown: Rename variables regex to REGEX for importing regex module. 2019-01-28 12:00:58 -08:00
Steve Howell 3b7d899532 bugdown: Use CompiledPattern in AtomicLinkPattern.
We avoid re-computing the regex string here, and we
also avoid re-compiling the regex itself.

I decided to put the "one_time" decorator in the
bugdown file itself, just to reduce friction in
folks reading the "buyer beware" comments.

Unfortunately, we can't use this for the
get_web_link_regex() function due to testing concerns,
so that continues to do an inelegant cache-with-global-var
scheme.
2019-01-28 11:58:47 -08:00
Steve Howell eea711a805 bugdown: Flatten get_web_link_regex().
We use early-exit to flatten the code.

I also tweaked the comments a bit based on some recent
profile findings.  (e.g. reading the file isn't actually
a big bottleneck, it's more the regex itself)
2019-01-28 11:58:46 -08:00
Steve Howell 852756aeb3 bugdown: Eliminate LinkPattern class.
The only code that used LinkPattern was AtomicLinkPattern.

We just move the helper method into AtomicLinkPattern.
2019-01-28 11:58:05 -08:00
Steve Howell 77446a710c bugdown: Optimize CompiledPattern.
We don't need the hacky step of passing in a blank
regex to the superclass's __init__ function.

All we need to do is assign `md` to `self.md`.
2019-01-28 11:57:28 -08:00
Steve Howell 025df33d7a bugdown: Rename VerbosePattern to CompiledPattern.
This class only requires a compiled regex--it's up
to the callers how they compile it (verbose or
otherwise).
2019-01-28 11:56:01 -08:00
Eeshan Garg 681f07f98f check_message: Refactor stream validation into separate function. 2019-01-28 11:48:17 -08:00
Eeshan Garg 4cb9a472e6 addressee: Add helper to check for missing or empty topic. 2019-01-28 11:42:26 -08:00
kunal-mohta b89769420a stream_color: Extend subscribe API to support color attribute.
This fixes an annoying bug where clicking to subscribe to a stream
would change the color shown in the "manage streams" UI immediately
after you click.

Fixes #11072.
2019-01-25 16:44:59 -08:00
Pragati Agrawal 1e811b42ec org settings: Add realm level setting for missed message content in email.
This adds a setting under "Notification" section of
"Organization settings" tab, which enables Organization administrator to
control whether the missed message emails include the message content or
not.

Fixes: #11123.
2019-01-25 14:34:10 -08:00
Vishnu Ks bec875a9af import realm: Use processes for resizing avatar images.
This should significantly improve the data import performance when
importing large open source realms from Slack.

Fixes #11009.
2019-01-25 12:37:12 -08:00
Tim Abbott a64c434b9e topics: Fix messages_for_topic to use iexact matching.
This fixes the archive codebase to use our covention of using
case-insensitive matching for message topics.
2019-01-24 17:55:15 -08:00
Vaibhav 268c25e95f streams: Prevent members from creating admin-only streams.
This a check on server side to verify whether the user sending request
to create stream where only admins can post is an admin or not; Raises
a JsonableError when the user is not the realm admin.
2019-01-24 10:51:45 -08:00
Steve Howell 0ef5d1f9c8 user status: Add "status_text" support in the backend.
You can now pass in an info field with a value
like "out to lunch" to the /users/me/status,
and the server will include that in its outbound
events.

The semantics here are that both "away" and
"status_text" have to have defined values in order
to cause changes.  You can omit the keys or
pass in None when values don't change.

The way you clear info is to pass the empty
string.

We also change page_params to have a dictionary
called "user_status" instead of a set of user
ids.  This requires a few small changes on the
frontend.  (We will add "status_text" support in
subsequent commits; the changes here just keep
the "away" feature working correctly.)
2019-01-24 09:46:35 -08:00
Steve Howell ac861f2b7d user status: Restructure code.
We now have single function that handle both away
and not-away.

This refactoring sets us up to piggyback "info" more
easily onto status updates.

The only thing that changes here is that we don't
delete database rows any more when users revoke
their away status.  Instead we just set the status
to NORMAL.
2019-01-24 09:46:35 -08:00
kunal-mohta c07f85250d messages: Extend do_delete_message to do_delete_messages.
do_delete_message has been renamed to do_delete_messages and all
occurrences of the function replaced with the appropriate new version.
2019-01-23 16:49:12 -08:00
kunal-mohta 67f2533ba3 messages: Rename do_delete_messages to do_delete_messages_by_sender. 2019-01-23 16:49:12 -08:00
Hemanth V. Alluri 6c18ca9cac validators: Add optional keys feature to check_dict and check_dict_only.
along with the `required_keys` argument an additional `optional_keys`
argument can be passed to both `check_dict` and `check_dict_only`.
Fixes #10892.
2019-01-22 10:03:21 -08:00
Harshit Bansal 3e3d8527b0 tests: Use `init_fakeldap()` in `test_signup.py`. 2019-01-17 10:16:48 -08:00
Harshit Bansal 356c5bfb0e dev_ldap: Make `userPassword` a multi-value attribute.
`fakeldap` assumes every attribute to be a multi-value attribute
while making comparison in `_comapare_s()` and so while making
comparisons for password it gives a false positive. The result
of this was that it was possible to login in the dev environment
using LDAP using a substring of the password. For example, if the
LDAP password is `ldapuser1` even entering `u` would log you in.
2019-01-17 10:16:48 -08:00
Rohitt Vashishtha 96aa1d4b37 markdown: Reduce mentions inside blockquotes to silent-mentions.
On the backend, we extend the BlockQuoteProcessor's clean function that
just removes '>' from the start of each line to convert each mention to
have the silent mention syntax, before UserMentionPattern is invoked.

The frontend, however, has an edge case where if you are mentioned in
some message and you quote it while having mentioned yourself above
the quoted message, you wouldn't see the red highlight till we get the
final rendered message from the backend.

This is such a subtle glitch that it's likely not worth worrying about.

Fixes #8025.
2019-01-16 16:08:37 -08:00
Rohitt Vashishtha 988af1c803 markdown: Use silent mentions in Notification Bot messages.
Messages converted:
- new stream
- rename stream
- new user signup
2019-01-16 16:01:06 -08:00
Rohitt Vashishtha f993fdd480 markdown: Add _@**Name** syntax for silent mentions.
These mentions look like regular mentions except they do not
trigger any notification for the person mentioned. These are
primarily to be used when you make a bot take an action and
the bot mentions you, or when you quote a message that mentions
you.

Fixes #11221.
2019-01-16 16:01:06 -08:00
Tim Abbott 1660856bf5 zoom: Fix handling of 201 status codes from zoom API.
Apparently, zoom's API will (sometimes?) return a 201 (not 200)
created in response to the API request to create a call.  We fix this
by using the proper requests check for whether or not the request
failed.
2019-01-16 15:37:06 -08:00
Harshit Bansal 5f76a65b1d emoji: Make unicode/span emojis more accessible.
This commit adds `aria-label="<title_text>"` and `role="img"` to
the generated HTML.

Fixes: #5975.
2019-01-16 09:07:19 -08:00
Anders Kaseorg 0a8598ce65 feedback: Generate the ticket number from the message ID.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-01-15 16:56:49 -08:00
Tim Abbott dfaa2e481d import: Log a warning when avatars can't be thumbnailed.
This fixes a potential crash in the import tool if a single user has a
broken avatar image.
2019-01-15 16:48:04 -08:00
Anders Kaseorg 601b5eb036 export: Avoid hardcoded paths in /tmp.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-01-15 16:05:51 -08:00
Hemanth V. Alluri 716bcad393 custom profile fields: Extract check_remove_user_custom_profile_value.
This moves the logic for deleting the user's custom profile field
value in the remove_user_custom_profile_data view function to a method
named check_remove_user_custom_profile_value in actions.py, so that we
can reuse it in the next commit.
2019-01-15 15:28:16 -08:00