Commit Graph

4018 Commits

Author SHA1 Message Date
Mateusz Mandera 3271235200 email_mirror: Ignore missed message email if the user isn't active. 2019-09-20 17:58:10 -07:00
Mateusz Mandera 21459ed193 test_email_mirror: Test missed message emails in a single ZulipTestCase.
With the way these tests are, it's unnecessary to have 3 separate
classes, and it makes it confusing to decide where to add a potential
additional mm email test.
2019-09-20 17:58:10 -07:00
Tim Abbott c19b7ec89d test_events: Remove unnecessary check_none validator.
This can be done cleanly using the existing equals(None).
2019-09-20 11:51:50 -07:00
Anders Kaseorg fbc2de157e templates: Move page_params to a <div> at the bottom of <body>.
In a gigantic realm where we send several MB of `page_params`, it’s
slightly better to have the rest of the `<body>` available to the
browser earlier, so it can show the “Loading…” spinner and start
fetching subresources.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-19 12:38:10 -07:00
Anders Kaseorg fc0554fcb8 emoji: Finish removing leftover code from banned Emoji sets.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-19 12:08:52 -07:00
Vishnu Ks c4af0b7bc4 mattermost: Support importing messages without team name.
Mattermost doesn't place private messages within a particular team,
which is what this is needed for.
2019-09-18 11:57:37 -07:00
Vishnu Ks bf5f531e90 import_util: Support huddles in SubscriberHandler. 2019-09-18 11:53:13 -07:00
Rishi Gupta 2646d70cbd emails: Add realm name to footer of missed message emails.
Missed message emails for mentions come from streams and possibly orgs you
don't normally get missed message emails for, so they can be hard to place.
2019-09-17 17:45:59 -07:00
Anders Kaseorg 7494f1600c templates: Move page_params from an inline script to the <body> dataset.
This sidesteps tricky escaping issues, and will make it easier to
build a strict Content-Security-Policy.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-17 16:06:33 -07:00
Wyatt Hoodes d2c3ddc765 test_events: Use stdout_suppressed in export test. 2019-09-13 11:54:14 -07:00
Wyatt Hoodes e37c91d195 test_realm_export: Use stdout_suppressed in tests.
From a testing perspective, calling `export_realm_wrapper`
prints unnecessary data to stdout.
2019-09-13 11:54:14 -07:00
Wyatt Hoodes e3595e4b3f test_slack_message_conversion: Use subTest for fixtures test. 2019-09-13 11:54:14 -07:00
Wyatt Hoodes dbaf6ac7e7 test_push_notifications: Remove fixtures print statement. 2019-09-13 11:54:14 -07:00
Wyatt Hoodes 4a64a487d3 bugdown: Remove print statement from fixtures test. 2019-09-13 11:54:14 -07:00
Wyatt Hoodes 698b0f1b91 test_management_commands: Use subTest with help tests.
We remove the print statement, and instead use
subTest to improve debuggability.
2019-09-13 11:54:14 -07:00
Thomas Ip 574c35c0b8 markdown: Render ordered lists using <ol> markup.
This brings us in line, and also allows us to style these more like
unordered lists, which is visually more appealing.

On the backend, we now use the default list blockprocessor + sane list
extension of python-markdown to get proper list markup; on the
frontend, we mostly return to upstream's code as they have followed
CommonMark on this issue.

Using <ol> here necessarily removes the behaviour of not renumbering
on lists written like 3, 4, 7; hopefully users will be OK with the
change.

Fixes #12822.
2019-09-08 16:42:20 -07:00
Tim Abbott 7e75f987df ldap: Fix logging of warning for deactivated users.
Also cleans up the interface between the management command and the
LDAP backends code to not guess/recompute under what circumstances
what should be logged.

Co-authored-by: mateuszmandera <mateusz.mandera@protonmail.com>
2019-09-08 09:35:23 -07:00
Tim Abbott d1a2784d52 ldap: Fix attempting to sync data for deactivated users.
The order of operations for our LDAP synchronization code wasn't
correct: We would run the code to sync avatars (etc.) even for
deactivated users.

Thanks to niels for the report.

Co-authored-by: mateuszmandera <mateusz.mandera@protonmail.com>
2019-09-08 09:35:23 -07:00
Mateusz Mandera 2ce2024bd7 ldap: Fix unintended user deactivation in case of connection failure.
Fixes #13130.

django_auth_ldap doesn't give any other way of detecting that LDAPError
happened other than catching the signal it emits - so we have to
register a receiver. In the receiver we just raise our own Exception
which will properly propagate without being silenced by
django_auth_ldap. This will stop execution before the user gets
deactivated.
2019-09-05 11:59:20 -07:00
Mateusz Mandera dfd6771237 stream settings: Use the .show-sender version of email address.
As requested in #13134.
2019-09-05 11:48:32 -07:00
Mateusz Mandera 307d8d8758 test_email_mirror: Better document the various address formats. 2019-09-05 11:48:32 -07:00
Vishnu KS 30440cf466 mattermost: Fix test flake in test_convert_channel_data.
So the reason 38f8cf612c seems
to be flaking is because the value of harry id switches between
1 and 2 in Xenial while in Bionic it would be fixed at 2. The
reason behind this is that Bionic ships with Python3.6 which
preserves dict insert order while Python3.5 that ships with Xenial
dont preserve the order. In initialize_stream_membership_dicts
we iterate user_data_map dict and the order in which the iteration
happens affects the ID of the users.
2019-09-05 11:20:33 -07:00
Wyatt Hoodes fc2574fdfd test_mattermost_importer: Add test coverage.
This is a rebase from a recent revert. A test
was failing non-deterministically relating to
`convert_channel_data` and use of an incorrect
`realm_id`.
2019-09-03 12:36:41 -07:00
Mateusz Mandera a7ec1a089c errors: Fix infinite recursion during memcached outages.
We have a very useful piece of code, _RateLimitFilter, which is
designed to avoid sending us a billion error emails in the event that
a Zulip production server is down in a way that throws the same
exception a lot. The code uses memcached to ensure we send each
traceback roughly once per Zulip server per 10 minutes (or if
memcached is unavailable, at most 1/process/10 minutes, since we use
memcached to coordinate between processes)

However, if memcached is down, there is a logging.error call internal
to the Django/memcached setup that happens inside the cache.set() call,
and those aren't caught by the `except Exception` block around it.

This ends up resulting in infinite recursion, eventually leading to
Fatal Python error: Cannot recover from stack overflow., since this
handler is configured to run for logging.error in addition to
logging.exception.

We fix this using a thread-local variable to detect whether we are
being called recursively.

This change should prevent some nasty failure modes we've had in the
past where memcached being down resulted in infinite recursion
(resulting in extra resources being consumed by our error
notifications code, and most importantly, the error notifications not
being sent).

Fixes #12595.
2019-09-03 11:44:39 -07:00
Tim Abbott 70c513a640 analytics: Fix logging for errors connecting to push bouncer.
There's no reason for this to be a category of error that emails the
server administrator, since there's a good chance that fixing it will
need to be done in the Zulip codebase, not administrator action.
2019-09-02 18:47:10 -07:00
Mateusz Mandera d70e1bcdb7 settings: Add FAKE_EMAIL_DOMAIN setting.
Fixes #9401.

This adds a FAKE_EMAIL_DOMAIN setting, which should be used if
EXTERNAL_HOST is not a valid domain, and something else is needed to
form bot and dummy user emails (if email visibility is turned off).
It defaults to EXTERNAL_HOST.

get_fake_email_domain() should be used to get this value. It validates
that it's correctly set - that it can be used to form valid emails.

If it's not set correctly, an exception is raised. This is the right
approach, because it's undesirable to have the server seemingly
peacefully operating with that setting misconfigured, as that could
mask some hidden sneaky bugs due to UserProfiles with invalid emails,
which would blow up the moment some code that does validate the emails
is called.
2019-08-30 14:59:00 -07:00
Anders Kaseorg c3e395b7d8 search: Reimplement ts_locs_array in pure PostgreSQL.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-28 17:59:12 -07:00
Anders Kaseorg d2d4800b1c test_narrow: Compile queries with the current SQLAlchemy dialect.
We’re about to start using PostgreSQL-specific syntax that can’t be
stringified without a specified dialect.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-28 17:58:01 -07:00
Yashashvi Dave e05429b6a9 custom fields: Add default external account custom fields. 2019-08-28 15:35:53 -07:00
Anders Kaseorg e8ee71e9a2 restore-backup, test_docs: Import settings from django.conf.
A settings module isn’t supposed to be imported directly.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-28 15:33:16 -07:00
Tim Abbott 89aeefed76 urls: Tighten many unintentionally broad URL patterns.
Previously, several of our URL patterns accidentally did not end with
`$`, and thus ended up controlling just the stated URL, but actually a
much broader set of URLs starting with it.

I did an audit and fixed what I believe are all instances of this URL
pattern behavior.  In the process, I fixed a few tests that were
unintentionally relying on the behavior.

Fixes #13082.
2019-08-26 20:51:49 -07:00
Tim Abbott e0f8228d6e auth: Add a test for the legacy /accounts/login/google/ mobile flow.
This is needed to return 100% URL coverage, and should also help
ensure we don't accidentally break the fix from
a43b231f90.
2019-08-26 20:51:49 -07:00
Tim Abbott 0ed0bb6828 messages: Add email/push notifications for wildcard mentions.
Historically, Zulip's implementation of wildcard mentions never
triggered either email or push notifications, instead being limited to
desktop notifications and the "mentions" counter.

We fix this just by plumbing the "wildcard_mentioned" flag through our
system.

Implements much of
https://github.com/zulip/zulip/issues/6040#issuecomment-510157264.
We're also now ready to seriously work on #3750.
2019-08-26 14:39:53 -07:00
Vishnu KS 01a51c8f4e slack: Rename added_recipient to slack_recipient_name_to_zulip_recipient_id. 2019-08-26 14:10:19 -07:00
Vishnu KS 9d51a1b527 slack: Rename added_users to slack_user_id_to_zulip_user_id. 2019-08-26 14:10:19 -07:00
Vishnu Ks 1e5c49ad82 slack: Support importing shared channels. 2019-08-26 14:10:19 -07:00
Vishnu Ks e09a29f4d3 slack: Refactor get_slack_api_data to accept multiple query params. 2019-08-26 14:10:19 -07:00
Tim Abbott b8bfdebea1 onboarding: Mark the most recent 20 messages as unread.
After a new user joins an active organization, it isn't obvious what
to do next; this change causes there to be recent unread messages in
the stream sidebar for the user to click on to get a feel for what's
happening in the organization and experiment with Zulip.

Fixes #6512.
2019-08-26 13:51:52 -07:00
Rohitt Vashishtha 8b443a25b8 markdown: Show link href if title is empty.
Fixes #6221.
2019-08-25 21:36:42 -07:00
Yashashvi Dave 330d065a69 zerver/tests/test_custom_profile_data: Group tests cases. 2019-08-25 20:59:13 -07:00
Yashashvi Dave ed5cb6c28b custom profile fields: Extract field validation function. 2019-08-25 20:59:08 -07:00
Tim Abbott 5e19546c14 messages: Fix unread_msgs accounting of wildcard mentions.
Previously, the unread_msgs data structure accounting (used for both
the web and mobile apps to determine the "Unread mentions" count
displayed in the UI) did not include wildcard mentions at all.

We fix this by adding the logic required to include properly that
data, with tests.  As discussed in #6040, it makes sense to include
muted streams and topics for the purpose of this calculation.

Fixes part of #6040.
2019-08-25 20:32:12 -07:00
Tim Abbott 4e65f1dd2b test_event_queue: Clean up testing setup.
Rather than continually resetting the contents of an existing event
queue, we allocate a new one for each subtest.

We also fix a rather confusing bundle of comments.
2019-08-25 19:37:08 -07:00
Rishi Gupta e058558a52 emails: Send invitation reminder email two days before expiry.
Hopefully this does a better job of spurring people to action, and also
suggests a self-service fix if they don't (i.e. contacting the person that
invited them).
2019-08-23 12:53:11 -07:00
Mohit Gupta e5482adec0 search: Add streams:public to search entire history of public streams.
Add ability to search entire message history of all public streams at
once. It includes all subscibed, non subscribed public streams messages
and even historical public stream messages sent before user had joined
an organization or stream.

Fixes #8859.
2019-08-22 13:40:49 -07:00
Mateusz Mandera c437fd4344 test_users: Eliminate hard-coded gravatar urls.
Instead of having a hard-coded url, it seems better to replace it with
get_gravatar_url - which returns the correct url, without breaking if
the email/id of the example user changes.
2019-08-21 21:28:09 -07:00
Mateusz Mandera 55bf13db88 test_user_groups: Eliminate hard-coded user ids. 2019-08-21 21:28:09 -07:00
Mateusz Mandera 52d4583987 test_push_notifs: Eliminate hard-coded user ids. 2019-08-21 21:28:09 -07:00
Mateusz Mandera 0edc7d74ff test_events: Eliminate hard-coded user id. 2019-08-21 21:28:09 -07:00
Mateusz Mandera afc9650ebb test_email_change: Eliminate hard-coded user id. 2019-08-21 21:28:09 -07:00