Commit Graph

9425 Commits

Author SHA1 Message Date
Tim Abbott 73655a6176 test_auth_backends: Disable Internet for httpretty.
This makes debugging issues when using httpretty a lot more
convenient.
2019-03-04 12:13:49 -08:00
Tim Abbott 6b796a1e8c test_auth_backends: Refactor some GitHub tests to be generic social.
This is preparation to being able to run these tests automatically
for the Google auth backend as well.
2019-03-04 12:13:11 -08:00
Harshit Bansal 216b7b0a19 auth: Remove `invalid_subdomain` restriction from LDAP backend.
Fixes: #11692.
2019-03-04 11:53:53 -08:00
Tim Abbott 16123c9a58 realm_logo: Fix synchronization of realm night logo.
The night logo synchronization on the settings page was perfect, but
the actual display logic had a few problems:

* We were including the realm_logo in context_processors, even though
  it is only used in home.py.
* We used different variable names for the templating in navbar.html
  than anywhere else the codebase.

* The behavior that the night logo would default to the day logo if
  only one was uploaded was not correctly implemented for the navbar
  position, either in the synchronization for updates code or the
  logic in the navbar.html templates.
2019-03-02 09:45:50 -08:00
Puneeth Chaganti 8d927eee2f digest: Fix incorrect remaining_unread_pms_count calculation. 2019-03-02 08:48:00 -08:00
Bennet Sunder 7c5f316cb8 alert_words: Performance improvements in looking for alert_words.
This commit leverages the ahocorasick algorithm to build a set of user_ids
that have their alert_words present in the message. It runs in linear time
of the order of length of the input message as opposed to number of
alert_words. This is after building a ahocorasick Automaton which runs
in O(number of alert_words in entire realm) which is usually cached.
2019-03-01 15:36:39 -08:00
overide 0dcfc22406 markdown: Fix numbered list handling of blank lines between blocks.
This fixes an issue where blank lines between blocks were causing
auto-numbering of list to stop before the blank line resulting
in two separate numbered list instead of one.

Edited significantly by tabbott to explain the tricky details in the
comments.

Fixes: #11651.
2019-03-01 15:29:07 -08:00
Rafid Aslam 9038230de8 decorator.py: Add `max_int_size` param to `to_non_negative_int()`.
Add `max_int_size` parameter to `to_non_negative_int()` in
decorator.py so it will be able to validate that the integer doesn't
exceed the integer maximum limit.

Fixes #11451
2019-03-01 11:16:46 -08:00
Tim Abbott 9693d8a651 docs: Extend testing discussion in events system doc.
Importantly, this also makes that section linkable.
2019-03-01 09:23:56 -08:00
Eeshan Garg 9252c43225 api/streams: Support including bot owner's subscriptions.
This is important for situations such as with our Zapier app,
where the requesting user may be a bot that would like to access
its owner's subscriptions.

Tweaked by tabbott to eliminate the 2^N growth of cases in
do_get_streams.
2019-02-28 22:32:05 -08:00
Eeshan Garg 85f453998e webhooks/zapier: Remove code related to our official Zapier app.
Our official Zapier app now uses our JS bindings (zulip-js) to
call the API directly and doesn't need to go through the webhook
anymore for anything.
2019-02-28 22:09:35 -08:00
Raymond Akornor 4603cdba7e tests: Optimize a slow test in tests_bots.py.
tests now ran in 7.649s from 9.297s.  And this test works just as well
with 3 bots, since only 3 database queries with 3 bots confirms we're
not doing linear queries in the number of bots in the organization.
2019-02-28 22:01:55 -08:00
Rishi Gupta 48eb0c2358 help: Update start-a-call to include Hangouts and Zoom. 2019-02-28 17:11:27 -08:00
Tim Abbott de65a04ae0 streams: Disable inline URL preview when rendering stream descriptions.
We want to use the baseline features of bugdown, but not fancy things
like inline URL previews, since the whole structure of stream
descriptions is to have a single-line thing supporting some
formatting.

The migration part of this change fixes a bug encountered by some
organizations upgrading from older versions of Zulip.
2019-02-28 17:00:40 -08:00
Tim Abbott d6c09eac51 bugdown: Add support for no_previews argument.
This allows us to have some features using bugdown rendering where
inline image previews will not be rendered (which would be problematic
for e.g. stream descriptions).
2019-02-28 16:54:04 -08:00
Tim Abbott 62dc6dda49 guests: Block guest user access to default streams.
Guest users will just get an empty list of default streams; we also
hide the "Default streams" organization view from the guest users UI.

This is for consistency with not providing guest users the full list
of streams in an organization.
2019-02-28 16:36:30 -08:00
Tim Abbott 1ce0e8256b zoom: Avoid sending Zoom API secret to other admin clients.
Fixing this involves fixing the backend to handle unchanged field
submissions of the Zoom credentials without trying to re-validate the
credentials (for performance) as well as to fetch the already-sent
secret.
2019-02-28 15:43:42 -08:00
Rishi Gupta d833c70dc7 org settings: Explain Zoom support is experimental, and fix a few strings.
Visually, #zoom_help_text acts like
.organization-settings-parent div:first-of-type when the Zoom option
is selected, but isn't treated as such.

No visual change with the #google_hangouts_domain change; just there to make
the code more readable/defensible.
2019-02-28 15:09:35 -08:00
Anders Kaseorg 88e90d34b9 try_git_describe: Set cwd, not --git-dir.
git describe --dirty requires a working tree, not just a repository.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-28 14:59:33 -08:00
Anders Kaseorg c6e60ebf0e backup: Set cwd while running pg_dump.
This avoids a spurious permission error inside the Postgres
`resolve_symlinks` function if we don’t have access to the current
working directory (e.g. we’re running with cwd /root inside `su
zulip`).

While we’re here, add a defensive `--` argument.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-28 14:40:28 -08:00
Tim Abbott bc3b864754 export: Add a bunch of comments to our export tool. 2019-02-28 12:20:08 -08:00
Tim Abbott de6f724bc5 middleware: Avoid doing work for statsd when not enabled.
This saves about 8% of the runtime of our total response middleware,
or equivalently close to 2% of the total Tornado response time.  Which
is pretty significant given that we're not sure anyone is using statsd
in production.

It's also useful outside Tornado, but the effect is particularly
significant because of how important Tornado performance is.
2019-02-27 17:53:15 -08:00
Tim Abbott c955b20131 middleware: Don't repreatedly regenerate open graph functions.
This avoids parsing these functions on every request, which was
adding roughly 350us to our per-request response times.

The overall impact was more than 10% of basic Tornado response
runtime.
2019-02-27 17:53:13 -08:00
Tim Abbott cefebd87a4 docs: Mention bug in ALWAYS_SEND_ALL_HOTSPOTS feature.
This should avoid some wasted time.
2019-02-27 10:21:30 -08:00
Rishi Gupta 6d51b25fc6 help: Rework docs for joining and inviting. 2019-02-27 09:33:37 -08:00
Tim Abbott 216d2ec1bf production: Add optional support for submitting usage statistics.
See documentation for details.
2019-02-26 17:35:10 -08:00
Greg Price de2f1ee0c4 push notif: Make O(1) database queries in handle_remove_... 2019-02-26 16:41:55 -08:00
Greg Price b0a84cd7ab message: Add an O(1)-query variant of bulk_access_messages.
We'll use this in the push-notifications code, in a context where
there should definitely already be UserMessage rows if everything's
gone normally... but explicitly checking at the top seems like the
right pattern from a secure-coding perspective.
2019-02-26 16:41:54 -08:00
Greg Price 9869153ae8 push notif: Send a batch of message IDs in one `remove` payload.
When a bunch of messages with active notifications are all read at
once -- e.g. by the user choosing to mark all messages, or all in a
stream, as read, or just scrolling quickly through a PM conversation
-- there can be a large batch of this information to convey.  Doing it
in a single GCM/FCM message is better for server congestion, and for
the device's battery.

The corresponding client-side logic is in zulip/zulip-mobile#3343 .

Existing clients today only understand one message ID at a time; so
accommodate them by sending individual GCM/FCM messages up to an
arbitrary threshold, with the rest only as a batch.

Also add an explicit test for this logic.  The existing tests
that happen to cause this function to run don't exercise the
last condition, so without a new test `--coverage` complains.
2019-02-26 16:41:54 -08:00
Greg Price 28ff9670de push notif: Push `gcm_options` logic inside "payload" helpers.
These are logically closely related.
2019-02-26 16:41:54 -08:00
Greg Price 8f26e12c85 push notif: Clarify get_*_payload, and factor another out.
This is a pure refactor; adding docstrings, making some names more
explicit, and pulling out one small helper.
2019-02-26 16:41:54 -08:00
Greg Price 69ded8b1b4 push notif: Drop irrelevant fields in `remove` payloads.
These fields don't make much sense in this case; and the client
doesn't look at them and never has.  Stop including them.
2019-02-26 16:41:54 -08:00
synicalsyntax 33bd52388a integrations: Rename social media logos to bypass Adblock Plus.
Adblock Plus's "Block social media icons tracking" setting blocked
integration logos for social media platforms from loading, so the logos
are renamed to bypass this.

Fixes #11590.
2019-02-23 10:56:43 -08:00
synicalsyntax d6483a99d0 integrations: Update xkcd logo. 2019-02-22 18:16:29 -08:00
Anders Kaseorg f1ec67c614 python: Mark intentionally unused imports with noop statements.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-22 16:54:47 -08:00
Anders Kaseorg 649235cfec python: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-22 16:54:36 -08:00
Eeshan Garg 8821b5a903 webhooks/pagerduty: Account for missing trigger summary data.
Some incoming payloads do not include details such as the trigger's
summary and description.
2019-02-22 15:59:28 -08:00
Eeshan Garg 912931e1bc webhooks/trello: Support updateCheckItemStateOnCard events. 2019-02-22 15:59:28 -08:00
Eeshan Garg 788dd48c93 webhooks/trello: Ignore createCheckItem events.
Notifications for every check item that is added to a card's
check list would be too noisy.
2019-02-22 15:59:28 -08:00
Eeshan Garg 9af2aa5566 webhooks/trello: Recommend wrapping webhook URL in quotes.
The ampersand in the query string of a URL can confuse bash,
leading to situations where anything in the URL beyond the
ampersand isn't parsed.
2019-02-22 15:59:28 -08:00
Challa Venkata Raghava Reddy e7fb19c8b0 invitations: Fix email validation errors for deactivated accounts.
This provides a much clearer error message when trying to invite a
user who has a deactivated account.

Fixes part of #8144.
2019-02-22 14:27:22 -08:00
Eeshan Garg cfe0f6b56d webhooks/clubhouse: Account for bug in GitHub branch payloads.
This change was prompted by a possible bug on Clubhouse's end. In
general, if a branch is added, it also prompts a workflow state
transition in its primary story.

However, our webhook error logs show an erroneous payload where
the same branch is added to another story, but there is no
workflow state transition. Also, both the stories are grouped in
the same payload, which confused/broke our code. Ideally, this
shouldn't happen and is most likely a bug on Clubhouse's end.
In most cases, changes included in Clubhouse payloads never
pertain to more than one parent entity (stories) simultaneously,
and we usually operate under the assumption that the changes
included therein are related to each other in terms of their
parent object (story or epic) and not a child object (the GitHub
branch).
2019-02-20 16:32:42 -08:00
Eeshan Garg 76b1a8379b webhooks/clubhouse: Support epic_archive events. 2019-02-20 16:32:42 -08:00
Eeshan Garg 51a1b76a1f webhooks/clubhouse: Ignore story comment updates.
Comment updates in general aren't very informative.
Plus, errors for this event type were cluttering up our
webhook logs.
2019-02-20 16:32:42 -08:00
Eeshan Garg 76e58d05cb webhooks/jira: Ignore a few noisy events.
UnexpectedWebhookEventType errors for these events were cluttering
up our webhook error logs.
2019-02-20 16:32:42 -08:00
Eeshan Garg fa29006311 webhooks/github: Ignore check_suite events.
A check suite is a collection of check runs. We care a lot more
about the outcomes of check runs in this case because check_run
payloads are a lot more informative than check_suite payloads.

(And in any case, the check_suite events are primarily for notifying
tools like CI to run checks).
2019-02-20 16:32:42 -08:00
Eeshan Garg a0717e4424 webhook/github: Support check_run events.
We only support notifications for events where a check run has
completed. Notifications for when a check run has been queued or
is in progress are not very informative and may be too noisy.
2019-02-20 16:32:39 -08:00
Tim Abbott cbc62b8e07 streams: Prevent creation of multi-line stream descriptions.
We do not anticipate our UI for showing stream descriptions looking
reasonable for multi-line descriptions, so we should just ban creating
them.

Given the frontend changes, multi-line descriptions are only likely to
show up from importing content from other tools, in which case
replacing newlines with spaces is cleaner than the alternative.
2019-02-20 12:28:00 -08:00
Rohitt Vashishtha 44ec83ef28 markdown: Render silent mentions as **name**.
This change should help people discover to distinguish
silent mentions in text as a part of Zulip syntax while
differentiating them from regular mentions.
2019-02-20 10:41:42 -08:00
Rohitt Vashishtha 57b9991396 markdown: Change syntax of silent mentions ( _@person -> @_person). 2019-02-20 10:41:42 -08:00