Commit Graph

21847 Commits

Author SHA1 Message Date
Steve Howell b0e844c676 refactor: Use get_avatar_field in message.py.
This is part of deprecating avatar_url_from_dict and
eventually supporting the client_gravatar field in
message-related requests from clients.
2017-10-12 14:00:41 -07:00
Steve Howell a6c3621f55 Support client_gravatar in /json/users endpoint. 2017-10-12 14:00:41 -07:00
Steve Howell 1fc6a5febc Add get_avatar_field() function.
This function is designed to replace avatar_url() and
avatar_url_from_dict() over time.

There are a few things new about it:

    * We make the parameters more explicit, rather than
      passing in an opaque dictionary or requiring a
      UserProfile object.  (A lot of our callers want
      to use `values()` for efficiency sake, since we
      are often doing bulk user operations.)

    * We start to support the client_gravatar option.
2017-10-12 14:00:41 -07:00
Tim Abbott 498d061e5e docs/email: Document how to use a Gmail account with 2FA.
Fixes #6974.
2017-10-12 11:38:01 -07:00
Tim Abbott aaf1cf848a docs/email: Clarify discussion Django's EMAIL_HOST_PASSWORD.
This fixes the second part of #6974 about EMAIL_HOST_PASSWORD.
2017-10-12 11:31:06 -07:00
derAnfaenger e1660462e1 linter: Add custom shebang rules section.
Tweaked by tabbott for clarity and to use print_err.
2017-10-12 11:21:38 -07:00
derAnfaenger 7deb698e02 email_log.py: Remove redundant shebang. 2017-10-12 11:18:45 -07:00
Brock Whittaker 6d8a8f2b7c list_render: Return data if no arguments present.
This will return the active data set if no arguments are provided to
the `data` method.
2017-10-12 10:42:06 -07:00
Steve Howell ae139a78b8 presence: Change "unknown" to "offline".
We never make an actionable distinction between the "unknown"
presence status and the "offline" status, so we now
just use "offline" as the status for persons who don't
have recent presence records that the client knows about.

(Usually, users without presence rows have never been online,
or they have been deactivated, or they have been offline so long
that they don't show up in our date-limited queries.)
2017-10-12 10:42:06 -07:00
Steve Howell 8593c25f45 refactor: Simplify user_last_seen_time_status().
We are about to stop supporting the presence status of "unknown."
Part of this fix is to stop checking for that status.

The implication of this change is that when we go
to display the time a user was last online, we now
mostly just look to see if presence.last_active_date
is undefined.  We were wary of that approach before, but it
is probably the most sane approach here.

I updated the comment abover this section to reflect
our philosophy going forward.

BTW the timestamp is kind of buried in the UI for now, as you have to
open the popover and then hover over the circular presence
indicator.
2017-10-12 10:42:06 -07:00
Steve Howell f0194f1821 Reset client descriptors at the start of event tests.
Fixes #6958
2017-10-12 10:27:16 -07:00
Steve Howell 941f718b08 Move tutorial code out of EventsRegisterTest.setUp.
Now we only run this code for the single test that needs
this setup.
2017-10-12 09:44:01 -07:00
Tim Abbott 6a5cb0e48c puppet: Make problems with Zephyr mirroring pageable.
Generally this indicates sending messages is completely broken.
2017-10-12 00:16:32 -07:00
Tim Abbott 5435fbf6c6 html_diff: Add missing mypy import.
It's getting really annoying that this isn't checked by our linter.
2017-10-12 00:13:58 -07:00
Tim Abbott 66a3f514d4 zilencer: Remove long-obsolete Deployment model.
This was a precursor to RemoteZulipServer, which is no longer used for
anything, and was cluttering the codebase.
2017-10-11 23:48:01 -07:00
Tim Abbott 339e206c90 highlight_html_differences: Improve logging output.
Now at least it will give the message ID, and thus be possible to
debug.
2017-10-11 23:38:29 -07:00
Tim Abbott 065033e864 auth: Fix mypy annotations in authenticate_remote_user. 2017-10-11 23:32:11 -07:00
Tim Abbott 57b8bfe84b authenticate_remote_user: Accept a realm object.
This paves the path for further code cleanup migrations.
2017-10-11 23:27:00 -07:00
Tim Abbott db645f5825 finish_google_oauth2: Depend on a realm existing. 2017-10-11 23:27:00 -07:00
Tim Abbott e48abcfcf1 auth: Fix confusing realm variable name in JWT auth. 2017-10-11 23:27:00 -07:00
Tim Abbott 51006f4a43 populate_db: Create bots before sending messages.
Apparently, the order we'd just created in
cbbdd539f4 broke populate_db; this is a
better and more sustainable fix to the overall problem.
2017-10-11 23:27:00 -07:00
Tim Abbott 6176d0fbca json: Replace most use of simplejson with json.
This is progress towards removing simplejson as a dependency.
2017-10-11 22:55:35 -07:00
Harshit Bansal f8b103bd98 tools: Add a test for checking locked requirements.
This commit adds a test to check if the user forgot to run
`tools/update-locked-requirements` after updating dependencies.

Modified by tabbott to disable it by default, since it takes over a
minute to run.

Fixes: #6324.
2017-10-11 22:55:15 -07:00
Harshit Bansal 0391619f48 tools: Modify `update-locked-requirements` to take optional arguments.
This commit modifies the `update-locked-requirements` tool to take
output file path as optional arguments.
2017-10-11 22:44:54 -07:00
Tim Abbott 539ff20ec2 socket: Add logging for CSRF issue in logging code.
This should help make it possible to debug #6961.
2017-10-11 22:39:14 -07:00
Tim Abbott cbbdd539f4 populate_db: Fix creating welcome-bot at the wrong place.
While it might be useful to have created welcome-bot earlier in a
certain sense, it's definitely not a good idea in this populate_db
implementation, because doing so threw off the random initial
assignment of users to streams and thus broke the casper tests.
2017-10-11 22:17:09 -07:00
Tim Abbott bfaf0fb3b2 build-release-tarball: Stop using .gitattributes.
Unfortunately, GitHub's web UI for generating release tarballs uses
`.gitattributes` to control what files to download, and thus if you
downloaded a source tarball for older Zulip versions using the GitHub
web UI, you'd be missing important files.

We fix this for future releases by moving the blacklist out of
.gitattributes.

Fixes #129.
2017-10-11 21:51:42 -07:00
Robert Hönig e749deb136 onboarding: Add welcome-bot response to initial user message.
Fixes #6030.
2017-10-11 20:45:42 -07:00
derAnfaenger 5ddc336844 tests: Add welcome bot as user. 2017-10-11 20:45:42 -07:00
Greg Price 55426894cd errors: Force a super-simpler handler for 400 errors.
This works around a bug in Django in handling the error case of a
client sending an inappropriate HTTP `Host:` header.  Various
internal Django machinery expects to be able to casually call
`request.get_host()`, which will attempt to parse that header, so an
exception will be raised.  The exception-handling machinery attempts
to catch that exception and just turn it into a 400 response... but
in a certain case, that machinery itself ends up trying to call
`request.get_host()`, and we end up with an uncaught exception that
causes a 500 response, a chain of tracebacks in the logs, and an email
to the server admins.  See example below.

That `request.get_host` call comes in the midst of some CSRF-related
middleware, which doesn't even serve any function unless you have a
form in your 400 response page that you want CSRF protection for.
We use the default 400 response page, which is a 26-byte static
HTML error message.  So, just send that with no further ado.

Example exception from server logs (lightly edited):

  2017-10-08 09:51:50.835 ERR  [django.security.DisallowedHost] Invalid HTTP_HOST header: 'example.com'. You may need to add 'example.com' to ALLOWED_HOSTS.
  2017-10-08 09:51:50.835 ERR  [django.request] Internal Server Error: /loginWithSetCookie
  Traceback (most recent call last):
    File ".../django/core/handlers/exception.py", line 41, in inner
      response = get_response(request)
    File ".../django/utils/deprecation.py", line 138, in __call__
      response = self.process_request(request)
    File ".../django/middleware/common.py", line 57, in process_request
      host = request.get_host()
    File ".../django/http/request.py", line 113, in get_host
      raise DisallowedHost(msg)
  django.core.exceptions.DisallowedHost: Invalid HTTP_HOST header: 'example.com'. You may need to add 'example.com' to ALLOWED_HOSTS.

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
    File ".../django/core/handlers/exception.py", line 109, in get_exception_response
      response = callback(request, **dict(param_dict, exception=exception))
    File ".../django/utils/decorators.py", line 145, in _wrapped_view
      result = middleware.process_view(request, view_func, args, kwargs)
    File ".../django/middleware/csrf.py", line 276, in process_view
      good_referer = request.get_host()
    File ".../django/http/request.py", line 113, in get_host
      raise DisallowedHost(msg)
  django.core.exceptions.DisallowedHost: Invalid HTTP_HOST header: 'example.com'. You may need to add 'example.com' to ALLOWED_HOSTS.
2017-10-11 20:43:36 -07:00
Tim Abbott 8828e96b87 presence: Avoid checking activity when reloading.
We sometimes get blueslip errors from browsers that are clearly still
attempting to reload long after they should have.  These browsers can
produce a lot of unnecessary presence update exceptions.

To solve that, we start checking reload_in_progress in the presence
code path.

While we're at it, we also add some blueslip logging for the reload
code path, in case it becomes useful when debugging future issues.
2017-10-11 20:39:28 -07:00
Brock Whittaker f8a2f06a84 reload: Continually attempt to reload page when reloading.
We've had a few reports of users using modern Chrome having problems
where reload.is_in_progress() was true, but the browser was just
sitting there, not having reloaded.

This will continually attempt to reload the page periodically try and
compensate for the behavior in Chrome where it appears that the tab
has to be active or semi-active for `location.reload` to be respected
when Chrome is trying to save power, which means that it should just
continually try until the page is active again, in which case the
`location.reload` func will work and reload the page.

See https://developers.google.com/web/updates/2017/03/background_tabs
for the Chrome featureset that we believe may be involved with this
issue.

Tweaked by tabbott to reload earlier and add the on-focus handler.

Fixes: #6821.
2017-10-11 20:38:33 -07:00
Tim Abbott 80e50f4b57 test-backend: Add goal of full coverage on zerver/*.py.
Like the zerver/lib changes, this should help make the backend
coverage TODO list a bit more explicit.
2017-10-11 20:17:37 -07:00
Tim Abbott ce579d5896 test-backend: Make test coverage gaps in zerver/lib more explicit.
This will make it much less likely that we'll have more regressions of
files losing 100% test coverage like push_notifications.py had
recently.
2017-10-11 20:17:37 -07:00
Tim Abbott 676a6af8cb push_notifications: Get back to 100% test coverage.
We didn't have tests for a few new code paths.
2017-10-11 20:17:27 -07:00
Tim Abbott 298c59f7fd push_notifications: Fix error message for unregistered bouncer.
Previously, we were just returning a JSON error to the client, when it
was a server problem.

Fixes #6639.
2017-10-11 19:09:24 -07:00
Tim Abbott 9e2e344937 report: Fix missing test coverage on draft sanitization. 2017-10-11 18:43:39 -07:00
Tim Abbott b3b5d5b7cd report: Avoid sending raw message content in error reporting.
This fixes a violation of Zulip's privacy policies (that error
reporting never contain message content) in the previous commit.
2017-10-11 17:44:05 -07:00
Harshit Bansal eedcbe2d47 drafts: Automatically delete misbehaving drafts while formatting.
While applying formatting to drafts if any draft contains some syntax
which our markdown processor is unable to process delete the draft so
that drafts overlay can be opened without any error. Also report the
exception to the server so that error can be fixed.
2017-10-11 17:44:03 -07:00
Steve Howell 9ecf41980c Remove message.is_mentioned in message events.
This field would get overwritten with an improper value when
we looped over multiple clients, due to not making full copies
of the message dictionary.  This failure would be somewhat
random depending on how clients were ordered in the loop.

The only consumers of this field were the mobile app and the
apply-events-to-unread-counts logic.  Both of these will now
use `flags` instead.
2017-10-11 16:55:34 -07:00
Steve Howell fed972d1fb Fix bug with applying message events to unread counts.
The `is_mentioned` flag in message events was buggy.  We now
look directly at flags.

We will kill off `is_mentioned` in a subsequent commit.

We also remove some debugging code in the test that was failing
before this fix.  The test would only fail when `is_mentioned`
was wrong, which never happened when you ran a single test, and
which would happen randomly when you ran multiple tests.
2017-10-11 16:55:34 -07:00
Tim Abbott 9426ee024b logging: Filter tornado.general logging for being spammy.
Apparently, this sockjs.tornado logging code resulted in a lot of
buggy error emails whenever a Zulip browser tried to reconnect on a
new IP.  I don't see an obvious way to suppress them from within
sockjs, but that might be a good follow-up issue.

Fixes #6959.
2017-10-11 16:53:52 -07:00
Brock Whittaker 36c2a6d8c9 streams: Change copy for stream creation suggestion.
This changes the copy and reverses the order of the button and text.
2017-10-11 16:17:36 -07:00
Brock Whittaker 71d61f93d5 streams: Change button styles on new stream creation form.
This changes the button styles from the old bootstrap buttons to
the new button styles on the stream creation form.
2017-10-11 16:17:36 -07:00
Brock Whittaker 6039687ecd streams: Add "Create Stream" button to right side.
This adds a button to allow a user to more easily discover the
stream creation functionality to the right side of the streams
overlay.
2017-10-11 16:17:36 -07:00
Steve Howell a6ad9a6d7c Add is_zephyr to the Stream model.
Add this field to the Stream model will prevent us from having
to look at realm data for several types of stream operations, which
can be prone to either doing extra database lookups or making
our cached data bloated.

Going forward, we'll set stream.is_zephyr to True whenever the
realm's string id is "zephyr".
2017-10-11 16:15:56 -07:00
Tim Abbott c1f12e3f8a scrolling: Fix out-of-order bug in the message list.
The issue has a lot of extra details, but in short, if several
messages were sent at very close to the same time, it's possible that
the event queues will receive the "new message" events out-of-order.
This, in turn, could cause `get_events` to return an incorrectly
sorted block of messages.  These would then be passed into
`message_list.add_messages`, which doesn't handle that sort of
unsorted situation correctly (in short, the `self.first.id()`
comparison checks are not accurate for that situation, since we don't
update the boundaries after the first messages is processed).

The end result of this bug was that it was possible for the message
list to be out-of-order, which in turn would cause exceptions when
scrolling with the mouse.

Fixes #6948.
2017-10-11 15:57:11 -07:00
Tim Abbott 9378489367 context_processors: Don't require coverage for ZulipElectron block.
This mostly makes sense because that code path doesn't do anything but
impact how a CSS class is handled.
2017-10-11 14:52:38 -07:00
amanharitsh123 39c0771800 portico: Add back to login page link on reset page.
This adds a link that goes back to the login page only for the
Zulip Electron application.

Fixes: #6763.
2017-10-11 14:02:49 -07:00
Brock Whittaker 5b3bd49061 portico: Enable content discrimination by platform.
This allows CSS to discriminate by platform and show particular
content; in this case showing things with the attribute
[if-zulip-desktop] content only on “ZulipElectron”.
2017-10-11 14:02:49 -07:00