Commit Graph

29462 Commits

Author SHA1 Message Date
Tim Abbott 1054d63820 message_fetch: Deduplicate logic for finish_newer_batch.
Like the other similar commits, we were doing the same work in all
code paths, just with a much more error-prone approach.

We can also now remove the now-unused finish_initial_narrow function.
2018-12-13 08:43:56 -08:00
Tim Abbott ce187b0899 message_fetch: Deduplicate logic for start_newer_batch.
Like the other commits in this series, we were already doing this in
all of the callers of load_messages; this centralizes that logic in a
less ad-hoc feeling way.

We no longer use or need the start_initial_narrow function.
2018-12-13 08:43:56 -08:00
Tim Abbott 7febf724f3 message_fetch: Deduplicate logic for start_older_batch.
We were doing this work individually in all the callers of
load_messages; better is to just do it in one place.
2018-12-13 08:43:56 -08:00
Tim Abbott 9ccb3a2ad1 message_fetch: Deduplicate logic for finish_older_batch.
Previously, each individual caller of load_messages that passed
num_before > 0 would do its own manual management of fetch_status;
now, we just do it inside load_messages.
2018-12-13 08:43:56 -08:00
Tim Abbott cd118bbc7e message_fetch: Track history_limited property. 2018-12-13 08:43:56 -08:00
Tim Abbott b8a45d889e message_fetch: Fix home_msg_list older FetchStatus tracking.
Apparently, the older side of the FetchStatus object for home_msg_list
was incorrectly not being maintained.  We got away with this, because
the do_backfill code path (which runs after we're done with the
load_more cycle) will correct the error for found_oldest.  But we
didn't have proper handling for history_limited here.
2018-12-13 08:43:56 -08:00
Tim Abbott ba6ef745fe message_fetch: Fix home_msg_list newer FetchStatus tracking.
When we're doing the load_more frontfill, we were not correctly
declaring that we were in the process of doing a fetch.  Because the
next load_more call clears this state anyway, this was generally a
short race, off-screen, but it is still a data flow bug.

See the upcoming commits for a refactor that will eliminate the
possibility of this sort of bug.
2018-12-13 08:43:56 -08:00
Tim Abbott a6ca95dfc4 slack import: Fix all messages being imported to one channel.
This was an ugly variable-escape-from-loop regression introduced in
e59ff6e6db.
2018-12-12 17:54:37 -08:00
Vishnu Ks 8a1794caa3 message: Store the value of first_visible_message_id in Realm table.
This eliminates a bunch of potentially buggy caching code, with no
material negative side effects.
2018-12-12 15:11:17 -08:00
Tim Abbott ee901ac8b1 django: Remove the Django contrib Sites app to fix 2FA QR codes.
Apparently, Django's get_current_site function (used, e.g., in
django-two-factor to look up the domain to use in QR codes) first
tries to use the Sites framework, and if unavailable, does the right
thing (namely, using request.get_host()).

We don't use the Sites framework for anything in Zulip, so the correct
fix is to just remove it.

Fixes #11014.
2018-12-12 15:01:55 -08:00
Tim Abbott 982a70431f message_list_view: Rerender mentions when user names change.
When a user's name is edited, currently we still show the old name is
mentions (though clicking on the item does the right thing).

However, at present, it creates a new problem in search results, where
the highlighting is removed by this substitution.
2018-12-12 13:38:12 -08:00
Tim Abbott d6217eb862 slack import: Fix empty values for custom profile fields.
The Slack import process would incorrectly issue
CustomProfileFieldValue entries with a value of "" for users who
didn't have a given CustomProfileField (especially common for the
"skype" and "phone" fields).  This had no user-visible effect, but
certainly added some clutter in the database.
2018-12-12 12:58:27 -08:00
Tim Abbott 5d5b1afd2d rate_limiter: Improve handling of deadlocks.
We should rate-limit users when our rate limiter deadlocks trying to
increment its count; we also now log at warning level (so it doesn't
send spammy emails) and include details on the user and route was, so
that we can properly investigate whether the rate-limiting on the
route was in error.
2018-12-12 12:19:01 -08:00
Tim Abbott e92e982342 docs: Add a troubleshooting note about cookies on root domain. 2018-12-12 12:08:25 -08:00
Tim Abbott 8a11c94a2d fakeldap: Add thumbnailPhoto/jpegPhoto for testing avatar syncing. 2018-12-12 11:23:33 -08:00
Tim Abbott b5e65a2ea0 fakeldap: Move fakeldap configuration into ZulipLDAPAuthBackendBase.
This allows us to use this for testing the ZulipLDAPUserPopulator code
as well.
2018-12-12 11:07:05 -08:00
Tim Abbott 33fb750275 fakeldap: Invert order of loop/conditionals.
This is about to save us some work when we add image attributes.
2018-12-12 10:46:54 -08:00
Tim Abbott e9900b2bdf gitter: Do something reasonable with invalid fullnames. 2018-12-12 10:07:52 -08:00
Tim Abbott 27f3919259 push_notifications: Also treat ConnectionError as a retry.
It's a bug that the HTTP20 library isn't catching these, but we just
have to work around it.
2018-12-12 09:40:51 -08:00
Tim Abbott 914ae88c62 push_notifications: Expand APNS main try/except block.
We should be catching exceptions trying to send the notice as well as
in the result section.
2018-12-12 09:40:51 -08:00
Anders Kaseorg 149132348f lint: Update ShellCheck to 0.6.0.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-12-12 08:24:01 -08:00
Roman Godov 9c8eeaed85 digest_email: Add endpoint for rendering digest to the web.
Adds "/digest/" endpoint for rendering content of digest email
to the web.

Fixes #9974
2018-12-11 13:38:30 -08:00
rht a0b0ccf972 provision: Do not require lsb_release for centos. 2018-12-11 13:08:27 -08:00
rht e0ec288928 parse_lsb_release: Add CentOS support. 2018-12-11 13:08:26 -08:00
rht c9d54f7854 puppet: Remove vendored puppetlabs apt and stdlibs dependencies.
This commit works by vendoring the couple functions we still use from
puppetlabs stdlib (join and range), but removing the rest of the
puppetlabs codebase, and of course cleaning up our linter rules in the
process.

Fixes #7423.
2018-12-11 13:03:26 -08:00
rht d2aa81858c puppet/zulip_ops: Replace apt::source with setup-apt-repo-debathena.
Tweaked by tabbott to use a clearer name.
2018-12-11 13:02:56 -08:00
rht 711b0fda8b scripts: Initialize setup-apt-repo-debathena. 2018-12-11 13:01:26 -08:00
rht 97766102df puppet/zulip: Replace apt::source and apt::ppa with setup-apt-repo. 2018-12-11 13:01:26 -08:00
Vishnu Ks c3aa7909ce docs: Update docs/git/ directory to reflect CI changes.
This isn't complete, in that we still need to rename the ReadTheDocs
articles (e.g.).
2018-12-11 12:46:18 -08:00
Tim Abbott 15d4b71e2e decorator: Skip rate limiting when accessing user uploads.
The code paths for accessing user-uploaded files are both (A) highly
optimized so as to not require a ton of work, and (B) a code path
where it's totally reasonable for a client to need to fetch 100+
images all at once (e.g. if it's the first browser open in a setting
with a lot of distinct senders with avatars or a lot of image
previews).

Additionally, we've been seeing exceptions logged in the production
redis configuration caused by this code path (basically, locking
failures trying to update the rate-limit data structures).

So we skip running our current rate limiting algorithm for these views.
2018-12-11 12:43:21 -08:00
Tim Abbott 9f8cc925b8 i18n: Update translation data from Transifex. 2018-12-11 11:31:00 -08:00
Tim Abbott b7127a395b server_settings: Use a reasonable default when User-Agent not specified.
Apparently, some Zulip clients don't send a User-Agent in their
requests to /server_settings, and this makes that not 500.
2018-12-11 11:30:11 -08:00
Tim Abbott 2a39852e91 auth: Add logging for which client does /server_settings requests.
This is generally helpful for debugging.
2018-12-11 11:30:11 -08:00
Tim Abbott 88368996fb decorator: Rename remote_server_request to skip_update_user_activity.
This makes it reasonable to call process_client from other contexts,
like /server_settings, where we don't have an authenticated
UserProfile object.
2018-12-11 11:30:11 -08:00
Tim Abbott 0d48f47eba scripts: Make setup-apt-repo a no-op if nothing has changed.
This performance optimization is important for being able to run this
from puppet in our production environment.
2018-12-10 17:36:28 -08:00
rht 05c4855cc9 circleci: Merge install dependencies step of trusty setup into the rest. 2018-12-10 17:06:37 -08:00
Steve Howell dfddb0a636 left sidebar: Remove unneeded overlay code.
These handlers are not reachable if overlays are
open.
2018-12-10 16:55:25 -08:00
Tim Abbott 14bfa74069 registration: Enable copying profile settings in production.
Now that we've styled this feature properly, this makes it possible to
copy various user-preferences type profile data in production when
making a new account with the same email address as an existing
account.
2018-12-10 16:55:07 -08:00
Tim Abbott 354109e5e9 casper: Fix test broken by recent signup refactor. 2018-12-10 16:55:07 -08:00
Max Nussenbaum 536de666df register: Style avatar that shows when importing settings.
This styles the avatar and username that show when the registering
user is importing their settings from an existing Zulip account.

Tweaked by tabbott to fix the test/linter failures, a bit of styling,
and tag strings for translation.
2018-12-10 15:40:30 -08:00
rht de9cefff8c circleci: Use CIRCLE_JOB env var for distro in cache filenames.
This lets us remove the ugly _dist.txt hack from our CircleCI
configuration.
2018-12-10 15:12:47 -08:00
Jack Zhang be9b6a6dee compatibility: Add a compatibility check to api_get_server_settings.
This should make it convenient for the mobile app to present errors of
the form "Your Zulip app is not new enough for this Zulip server".
2018-12-10 15:06:09 -08:00
Jack Zhang 8711fe7bae compatibility: Respond with HTTP 400 when no User-Agent header.
Prior to this commit, a request to /compatibility without a User-Agent
header would result in the server 500ing.
2018-12-10 15:04:34 -08:00
Vishnu Ks a86faecff5 tools: Rename tools/travis directory to tools/ci. 2018-12-10 11:07:26 -08:00
Tim Abbott a1d99adce7 auth: Handle SSO_APPEND_DOMAIN in remote_user SSO for mobile.
Apparently, while the main code path through
login_or_register_remote_user was correctly calling
remote_user_to_email(username) to get a proper email address for
situations where auth username != email (i.e. when SSO_APPEND_DOMAIN
is set), we neglected to do so in the mobile_flow_otp corner case.

Fixes #11005.
2018-12-10 10:36:15 -08:00
Aditya Jain 8f0ccdf1c9 user settings: Fix active background links when a modal is visible.
A bug caused background links to open even when a modal in the user
settings overlay was active in the foreground. This commit fixes this
by disabling mouse events for the background when the modal is active,
and restoring them as soon as the modal starts closing.

Fixes #10654.
2018-12-09 22:32:53 -08:00
Akash Nimare b90eeee1ec stream: Vertically align check icons on the left of streams list. 2018-12-09 22:25:36 -08:00
konraddeka e6be407e34 provision: Avoid spending time on update-authors-json.
This optimizes tools/provision by not running
`tools/update-authors-json --use-fixture` unless either the script
itself or its fixtures file (zerver/tests/fixtures/authors.json) was
changed.

Fixes #10991.
2018-12-09 22:21:51 -08:00
neiljp (Neil Pilgrim) 63768858ff dependencies: Upgrade mypy to version 0.650.
Broaden the type of the AbstractEnum __reduce_ex__ parameter to object; this
matches the parameter type specified in the latest enum.pyi file in typeshed.

Fixes #10996.
2018-12-09 22:20:12 -08:00
rht 9bfea2adba provision.py: Translate COMMON_APT_DEPENDENCIES to yum. 2018-12-09 21:54:02 -08:00