Commit Graph

39245 Commits

Author SHA1 Message Date
palash 34003fc7f6 email_mirror: Change root logger to zerver.lib.email_mirror. 2020-09-12 10:53:56 -07:00
palash 60f9f87c5d test_external: Refactor mock.patch to assertLogs.
Replaced mock.patch with assertLogs for testing log outputs
in file zerver/tests/test_external.py
2020-09-12 10:53:56 -07:00
palash 485d740fe8 apple auth: Change logger to zulip.auth.apple from root logger.
Replace default root logger with zulip.auth.apple for apple auth
in file zproject/backends.py and update the test cases
accordingly in file zerver/tests/test_auth_backends.py
2020-09-12 10:53:56 -07:00
palash 04f08f45e7 test_auth_backends: Refactor mock.patch to assertLogs.
Replaced mock.patch with assertLogs for testing log outputs
in file test_auth_backends.py.

This change requires adjusting
test_log_into_subdomain_when_email_is_none to use an explicit token
since that appears in the log output.
2020-09-12 10:53:37 -07:00
Alex Vandiver 1c17583ad5 puppet: Restrict postfix incoming addresses to postmaster and zulip.
This removes the possibility of local user enumeration via RCPT TO.
2020-09-11 18:49:22 -07:00
Aman Agrawal ed8796f23f report: Allow anonymous queries to report (un)narrow times. 2020-09-11 16:57:27 -07:00
Alex Vandiver 3f6e4ff303 webhooks: Move the extra logging information into a formatter.
This clears it out of the data sent to Sentry, where it is duplicative
with the indexed metadata -- and potentially exposes PHI if Sentry's
"make this issue public" feature is used.
2020-09-11 16:43:29 -07:00
Alex Vandiver b8a2e6b5f8 webhooks: Configure webhook loggers in zproject/computed_settings.py.
This limits the webhook errors to only go to their respective log
files, and not to the general server logs.
2020-09-11 16:43:29 -07:00
Alex Vandiver 006875e7d0 sentry: Add additional context (client, fallback realm) to Sentry tags. 2020-09-11 16:43:29 -07:00
Alex Vandiver 6323218a0e request: Maintain a thread-local of the current request.
This allows logging (to Sentry, or disk) to be annotated with richer
data about the request.
2020-09-11 16:43:29 -07:00
Mateusz Mandera bb4567f57e queue: Extract get_remaining_queue_size method. 2020-09-11 15:51:07 -07:00
Mateusz Mandera aae84197e8 check-rabbitmq-queue: Use list_queues output for current backlog size.
The value in the stats file can get outdated if the queue hasn't done
enough iterations to update the stats file for a while. The queue size
output by rabbitmqctl list_queues is more up to date, and empirically
tends to agree with the value in the stats file (when the stats file is
fresh).
2020-09-11 15:51:07 -07:00
Mateusz Mandera 1d466a4fc5 queue: Make embed_link updates stats on every iteration. 2020-09-11 15:51:07 -07:00
Tim Abbott f0c2c640b6 api docs: Add changes for community_topic_editing_limit_seconds.
The previous commit fixed this in the changelog, but it should have
also been documented directly as well.
2020-09-11 15:50:06 -07:00
Kaustubh Nair acf31f8934 docs: Add community_topic_editing_limit_seconds value to docs.
This was added to the response in e242ddc. Specifying the value
makes sure that the user does not have to dig in the commit history.
2020-09-11 15:48:08 -07:00
Thomas Güttler 57ad171112
docs: Remove typo (two periods at end of sentence). 2020-09-11 15:47:09 -07:00
Alex Vandiver 4917391133 webhooks: Derive payload from request itself. 2020-09-10 17:47:22 -07:00
Alex Vandiver a1f5f6502c webhooks: In logger, pull user from request, rather than parameter.
request.user is set by validate_api_key, which is called by
webhook_view and authenticated_rest_api_view.
2020-09-10 17:47:22 -07:00
Alex Vandiver d04db7c5fe webhooks: Remove repetitive argument to UnsupportedWebhookEventType.
The name of the webhook can be added by the webhook decorator.
2020-09-10 17:47:21 -07:00
Alex Vandiver e2ab7b9e17 webhooks: Update API_KEY_ONLY_WEBHOOK_LOG_PATH to WEBHOOK_LOG_PATH.
The existence of "API_KEY" in this configuration variable is
confusing.  It is fundamentally about webhooks.
2020-09-10 17:47:21 -07:00
Alex Vandiver cf6ebb9c8d webhooks: Rename api_key_only_webhook_view to webhook_view.
There are no other types of webhook views; this is more concise.
2020-09-10 17:47:21 -07:00
Alex Vandiver 8cfacbf8aa webhooks: Update comment about typing the webhook decorator.
The previous link was to "extended callable" types, which are
deprecated in favor of callback protocols.  Unfortunately, defining a
protocol class can't express the typing -- we need some sort of
variadic generics[1].  Specifically, we wish to support hitting the
endpoint with additional parameters; thus, this protocol is
insufficient:

```
class WebhookHandler(Protocol):
    def __call__(request: HttpRequest, api_key: str) -> HttpResponse: ...
```
...since it prohibits additional parameters.  And allowing extra
arguments:
```
class WebhookHandler(Protocol):
    def __call__(request: HttpRequest, api_key: str,
                 *args: object, **kwargs: object) -> HttpResponse: ...
```
...is similarly problematic, since the view handlers do not support
_arbitrary_ keyword arguments.

[1] https://github.com/python/typing/issues/193
2020-09-10 17:47:21 -07:00
Alex Vandiver ea8823742b webhooks: Adjust the name of the unsupported logger.
`zulip.zerver.lib.webhooks.common` was very opaque previously,
especially since none of the logging was actually done from that
module.

Adjust to a more explicit logger name.
2020-09-10 17:47:21 -07:00
Alex Vandiver 482c964dd3 puppet: Logrotate for webhook exceptions. 2020-09-10 17:47:21 -07:00
Alex Vandiver e38051736d puppet: Wrap and sort logrotate config. 2020-09-10 17:47:21 -07:00
Alex Vandiver 9ea9752e0e webhooks: Rename UnexpectedWebhookEventType to UnsupportedWebhookEventType.
Any exception is an "unexpected event", which means talking about
having an "unexpected event logger" or "unexpected event exception" is
confusing.  As the error message in `exceptions.py` already explains,
this is about an _unsupported_ event type.

This also switches the path that these exceptions are written to,
accordingly.
2020-09-10 17:47:21 -07:00
Alex Vandiver 8016769613 webhooks: Move UnexpectedWebhookEventType into zerver.lib.exceptions.
8e10ab282a moved UnexpectedWebhookEventType into
`zerver.lib.exceptions`, but left the import into
`zserver.lib.webhooks.common` so that webhooks could continue to
import the exception from there.

This clutters things and adds complexity; there is no compelling
reason that the exception's source of truth should not move alongside
all other exceptions.
2020-09-10 17:47:21 -07:00
Mateusz Mandera f95dd628bd email: Fix race conditions with concurrent ScheduledEmail handling.
The main race conditions, which actually happened in production was with
concurrent execution of deliver_email and clear_scheduled_emails.
clear_scheduled_emails could delete all email.users in the middle of
deliver_email execution, causing it to pass empty to_user_ids list to
send_email. We mitigate this by getting the list of user ids in a single
query and moving forward with that snapshot, not having to worry about
database data being mutated anymore.

clear_scheduled_emails had potential race conditions with concurrent
execution of itself due to not locking the appropriate rows upon
selecting them for the purpose of potentially deleting them. FOR UPDATE
locks need to be acquired to prevent simultaneous mutation.

Tested manually with some print+sleep debugging to make some races
happen.

fixes #zulip-2k (sentry)
2020-09-09 15:59:01 -07:00
Anders Kaseorg b7b7475672 python: Use standard secrets module to generate random tokens.
There are three functional side effects:

• Correct an insignificant but mathematically offensive bias toward
repeated characters in generate_api_key introduced in commit
47b4283c4b4c70ecde4d3c8de871c90ee2506d87; its entropy is increased
from 190.52864 bits to 190.53428 bits.

• Use the base32 alphabet in confirmation.models.generate_key; its
entropy is reduced from 124.07820 bits to the documented 120 bits, but
now it uses 1 syscall instead of 24.

• Use the base32 alphabet in get_bigbluebutton_url; its entropy is
reduced from 51.69925 bits to 50 bits, but now it uses 1 syscall
instead of 10.

(The base32 alphabet is A-Z 2-7.  We could probably replace all of
these with plain secrets.token_urlsafe, since I expect most callers
can handle the full urlsafe_b64 alphabet A-Z a-z 0-9 - _ without
problems.)

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-09 15:52:57 -07:00
Dinesh 56546170cf puppeteer: Save pid of `run-dev.py --test` in var/puppeteer.
Also replaces a reference to casper in a comment related to --test.
2020-09-09 13:38:39 -04:00
Dinesh ad0f0564b1 puppeteer: Rename build_for_prod_or_casper to build_for_prod_or_puppeteer. 2020-09-09 13:38:39 -04:00
Dinesh c64888048f puppeteer: Rename CASPER_TESTS env variable to PUPPETEER_TESTS.
Also modified few comments to match with the changes.
2020-09-09 13:38:39 -04:00
Dinesh f2b3afcd56 puppeteer: Mention about docs and screeshots on failure. 2020-09-09 13:38:39 -04:00
aryanshridhar 42b1ed23a9 reactions_tooltip : Improved responsiveness of tooltip for reactions.
Improved responsiveness of message reactions tooltip .
Added css property for the reaction tooltip in ./static/js/click_handlers.js so that the tooltip doesn't exceed the sidebar .

Adressed a comment in #15364 .
2020-09-09 13:24:04 -04:00
Dinesh cc4f35bfa8 puppeteer: Wait for save button before checking the text.
Found a flake that this caused with runs for the previous fix.
The waitForSelector was in a wrong order.
2020-09-09 13:14:16 -04:00
Dinesh 6dc2547336 puppeteer: wait for visibility before click -- admin test.
Wait for disable_stream_notifications selector to be visible
before clicking as it could cause flakes if the test tries
to click without it being visible.
2020-09-09 13:14:16 -04:00
Dinesh 1f62ba5750 puppeteer: Fix unsaved status selector waitFor timeout.
Added a stronger validation of waiting for text "Verona" to
appear but that didn't really seem to have worked though it
seemed like fixing the flake by passing ~600 runs.
So, change the puppeteer click to a click through evaluate
as we had experiences where page.click() didn't work sometimes.

Though this has passed 1000 runs on CI, I'm not very certain
if this fixed it as this test passed 1000 times with my previous
PR fixing the same flake.
2020-09-09 13:14:16 -04:00
Tim Abbott 7b62d31c32 invite: Set tabindex for 'Generate invite link'. 2020-09-04 13:03:27 -07:00
Abhijeet Prasad Bodas 49fd272d7d invite: Fix compose box focus on clicking (Un)check All in invite window.
Fixes #16077.
2020-09-04 12:58:11 -07:00
Abhijeet Prasad Bodas e5054a3a66 invite: Scroll to top and focus email input after submit in invite.js.
Part of #16077.
2020-09-04 12:58:11 -07:00
Abhijeet Prasad Bodas b570cf1433 invite: Allow submitting invite form with Ctrl+Enter in invite.js.
Part of #16077.
2020-09-04 12:58:11 -07:00
Abhijeet Prasad Bodas 9fc10479fe invite: Focus first input in invite window in invite.js.
Part of #16077.
2020-09-04 12:58:11 -07:00
gr455 a8511bc710 docs: Add alt text to images
Add alt text to images in docs roughly stating the contents of the
image.
2020-09-04 12:56:07 -07:00
Vinit Singh 140d24ca7f compose: Update compose placeholder text if stream name is changed.
Compose box placeholder text for streams currently updates when focus
is shifted to the text area.

With this change, it will also get updated when the stream name is
changed (it already updates if topic names are changed).
2020-09-04 12:45:51 -07:00
Vinit Singh d43b1765d5 compose: Update compose placeholder text if recipients are changed.
Currently, compose box placeholder text for PMs only gets updated
when the focus shifts to it.
With this change, the text is now also updated if recipients are
added or removed.

Fixes #15897.
2020-09-04 12:45:51 -07:00
Vinit Singh c6e278ab5a input_pills: Fix `onPillCreate` to update when adding pills with typehead.
Previously, onPillCreate function was called after the individual
pill object was created.
Now, we call it after creating and adding it to the pill container.
2020-09-04 12:45:51 -07:00
Aman Agrawal be7238afca tooltip: Fix arrow position.
This was one of the regressions from our upgrade to bootstrap 2.3.2.
2020-09-04 10:36:02 -07:00
Aman c114447e11 topic_history: Allow anonymous access in web-public streams.
For web-public streams, clients can access full topic history
without being authenticated. They only need to additionally
send "streams:web-public" narrow with their request like all
the other web-public queries.
2020-09-04 10:17:56 -07:00
Anders Kaseorg 3b301f522b python: Tweak some magic trailing commas to avoid Black bugs.
https://github.com/psf/black/issues/1658
https://github.com/psf/black/issues/1671

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-03 17:51:09 -07:00
Anders Kaseorg f91d287447 python: Pre-fix a few spots for better Black formatting.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-03 17:51:09 -07:00