Commit Graph

2961 Commits

Author SHA1 Message Date
Tim Abbott c7b7893254 auth: Give nicer subdomain errors when using ZulipDummyBackend.
This improves Google and JWT auth as well as the registration
codepath to log something if the wrong subdomain is encountered.

Ideally, we'd have tests for these, and code to make the Google and JWT
auth cases show a clear error message.
2016-09-27 23:25:07 -07:00
Tim Abbott b5b00e3a36 subdomains: Enforce subdomain checks on every API endpoint.
This ensures that everything is using the correct subdomain for
requests.  While it probably wouldn't be a real security problem for
the wrong subdomain to work, this enforcement is essential to catching
bugs in the product and users' API scripts.
2016-09-27 23:25:07 -07:00
Tim Abbott a4e5450ace subdomains: Add support for aliases of the root subdomain.
We default to counting "www" as such an alias.
2016-09-27 23:25:07 -07:00
hackerkid ea39fb2556 Add option for hosting each realm on its own subdomain.
This adds support for running a Zulip production server with each
realm on its own unique subdomain, e.g. https://realm_name.example.com.

This patch includes a ton of important features:
* Configuring the Zulip sesion middleware to issue cookier correctly
  for the subdomains case.
* Throwing an error if the user tries to visit an invalid subdomain.
* Runs a portion of the Casper tests with REALMS_HAVE_SUBDOMAINS
  enabled to test the subdomain signup process.
* Updating our integrations documentation to refer to the current subdomain.
* Enforces that users can only login to the subdomain of their realm
  (but does not restrict the API; that will be tightened in a future commit).

Note that toggling settings.REALMS_HAVE_SUBDOMAINS on a live server is
not supported without manual intervention (the main problem will be
adding "subdomain" values for all the existing realms).

[substantially modified by tabbott as part of merging]
2016-09-27 23:24:14 -07:00
Tim Abbott 15f6cc7c84 validate_api_key: Accept the request as an argument.
This is a prerequisite for checking the subdomain of the request.
2016-09-27 21:18:29 -07:00
Tim Abbott bbab3cdc30 test_helpers: Add HostRequestMock helper class. 2016-09-27 21:18:29 -07:00
Tim Abbott 8442e9249c Reflow annotation for generic_bulk_cached_fetch.
This is a test of mypy's new support for annotating functions that
take lots of arguments.
2016-09-27 20:36:56 -07:00
Brock Whittaker 98937ac539 bugdown: Set "title" attribute to actual image title when available.
This sets the “title” attribute on the image to the actual title of
the image specified by the user in their markdown, rather than just
the URL of the full link to it.
2016-09-27 17:34:05 -07:00
Tim Abbott 63f92adfbb RateLimitTests: Remove now-unnecessary API key logic. 2016-09-27 14:54:10 -07:00
Tim Abbott 03bf8893e2 Remove legacy /api/v1/send_message endpoint.
This was the original way to send messages via the Zulip API in the
very early days of Zulip, but was replaced by the REST API back in
2013.

Fixes: #730.
2016-09-27 14:51:54 -07:00
Steve Howell edfa022bac Remove json/get_active_statuses endpoint. 2016-09-27 14:33:56 -07:00
Umair Khan 76c8830f4a Test unicode file name. 2016-09-26 21:13:34 -07:00
Umair Khan fd9dd51d16 Url encoded name of the file should be an ascii.
The url encoded name of the file should not be a unicode. This
results in an error when we later try to unquote it.

Fixes: #1803
2016-09-26 21:13:34 -07:00
Sahil Dua 058587da77 Remove extra new lines at the ends of Zulip authoried files.
Fixes #1627.

[tweaked by tabbott to avoid patching third-party modules, for now]
2016-09-26 21:05:24 -07:00
Tim Abbott 49faad7bb1 GitLab: Fix minor bytes/text types issue. 2016-09-25 23:29:36 -07:00
Tim Abbott a3eb0be52a GitLab: simplify topics in GitLab integration. 2016-09-25 23:12:24 -07:00
Supriya 4959e8d10a tests: Extract and use assert_max_length test helper.
This makes it much more explicit in which cases we're checking
equality or a maximum in our use of `assert_length`.

Fixes #1400.
2016-09-25 15:44:58 -07:00
Steve Howell 1672353558 comments: Add comment to Realm.deployment().
I refer to github issue 1845 here.
2016-09-24 16:56:34 -07:00
Steve Howell bc9a75ab24 Remove unreachable remove_unreachable() method.
Sorry, couldn't resist wording the commit message like that. :)

The remove_unreachable() method on Message was no longer being
used, and the commit history made it fairly clear we won't need it
in the future.
2016-09-24 16:56:34 -07:00
Steve Howell 98a500d46f tests: Add test_stream_message_unicode(). 2016-09-24 16:56:34 -07:00
Steve Howell 5486e539fd tests: Test Message.__unicode__() for PMs. 2016-09-24 16:56:34 -07:00
Steve Howell a79a627166 tests: Add more cases to test_terms_of_service().
This gives us more test coverage on UserProfile.major_tos_version().
2016-09-24 16:56:34 -07:00
Steve Howell dfa416cfc8 Remove last_reminder_tzaware().
We can now rely on UserProfile.last_reminder being time zone
aware, or even if it isn't, it's a self-correcting problem the
first time a reminder is sent.  (It's a non-problem to be off
by a few timezones if somebody still has an old value there, because
they will still be outside the 1-minute nag window even with the
timezone disparity.)
2016-09-24 16:56:34 -07:00
Steve Howell 0dddb9c877 tests: Add test_flush_realm_filter(). 2016-09-24 16:56:34 -07:00
Steve Howell 6673b28cde Extract domain_in_local_realm_filters_cache().
This is partly a concession to testing; it's really hard to test
that we are flushing the cache properly if tests need to look
at a global variable in models.py that can be re-assigned on every
request.  Extracting this function makes it easy for tests to know
whether a domain is in the local cache.
2016-09-24 16:56:34 -07:00
Steve Howell d28c3d08a8 tests: Add test_maybe_update_realm_filters(). 2016-09-24 16:56:34 -07:00
Steve Howell e0dc6092d5 tests: Call flush_per_request_caches() in test_realm_patterns().
This is a slightly more realistic test, and it adds some test
coverage.
2016-09-24 16:56:34 -07:00
Rishi Gupta 03ce3e5fa4 test_signup.py: Refactor getting confirmation_urls to test_helpers.py
It is a bit of tricky / scary looking code that was repeated several times
in test_signup.
2016-09-23 15:44:28 -07:00
Rishi Gupta 4b62ee8f38 initial invite page: Fix email validation when not restricted_to_domain.
We currently do
  var invite_suffix = "{{invite_suffix}}";
in javascript in the initial_invite_page.html template.
This sets invite_suffix to "{{invite_suffix}}" when the template is rendered
without invite_suffix in the params, rather than to "" as intended. This
later causes problems in the invite_email validator in initial_invite.js.
2016-09-23 15:44:28 -07:00
Steve Howell a04a095738 Speed up alert word detection during message sends.
We no longer use all the alert words for all the users in the
entire realm when we look for alert words in a newly sent/edited
message.  Now we limit the search to only all the alert words
for all the users who will get UserMessage records.  This will
hopefully make a big difference for big realms where most messages
are only sent to a small subset of users.
2016-09-23 15:24:55 -07:00
Steve Howell 40b18094ec alert words: Refactor alert word detection.
The bugdown parser no longer has a concept of which users need which
alert words, since it can't really do anything actionable with that info
from a rendering standpoint.

Instead, our calling code passes in a set of search words to the parser.
The parser returns the list of words it finds in the message.
Then the model method builds up the list of user ids that should be
flagged as having alert words in the message.

This refactoring is a little more involved than I'd like, but there are
still some circular dependency issues with rendering code, so I need to
pass in the rather complicated realm_alert_words data structure all the way
from the action through the model to the renderer.

This change shouldn't change the overall behavior of the system, except
that it does remove some duplicate regex checks that were occurring when
multiple users may have had the same alert word.
2016-09-23 15:21:17 -07:00
Steve Howell cb0d75b23b rendering: Consolidate code to render new messages.
We now use render_incoming_message() to render all incoming
new messages (sends/edits), so that they will get the same treatment.

This change also establishes do_send_messages() as the code
path to get new messages rendered.  It removes some
logic from check_message() that only happened on certain code paths
for sending messages, and which would only detect failures by
expensively rendering messages, so it wasn't much of a guard.

This change also helps to phase out maybe_render_content(), which
deepens the call stack without providing much clarity to the reader,
since it's behavior is so variable.

Finally, this sets up to fix a flaw in the way we compute which
users have alert words in their messages (in a subsequent commit).
2016-09-23 15:21:17 -07:00
Tomasz Kolek 95825973c7 Add branch name to topic in Gitlab integration.
Fixes: #1831.
2016-09-23 10:04:01 -07:00
Tomasz Kolek 8cdc5b7a02 Fix bug with wrong branch name in Gitlab integration.
Need to change lstrip to replace particular string with empty string.

Fixes: #1830.
2016-09-23 10:03:09 -07:00
Tomasz Kolek d4da60fbe2 Split test_hooks file into one test file per hook. 2016-09-23 10:02:16 -07:00
Tomasz Kolek 9197c82d8f Move WebhookTestCase class from test_hooks.py to test_helpers.py. 2016-09-23 10:02:16 -07:00
Steve Howell f0eaee68e4 bug: Fix traceback in get_missed_message_token_from_address().
If you supplied an unrecognizable address to our email system,
or you had EMAIL_GATEWAY_PATTERN configured wrong,
the get_missed_message_token_from_address() used to crash
hard and cryptically with a traceback saying that you can't
call startswith() on a None object.

Now we throw a ZulipEmailForwardError exception.  This will
still lead to a traceback, but it should be easier to diagnose
the problem.
2016-09-22 13:41:26 -07:00
Steve Howell dbbc64dbfe bug: Fix code that mis-identifies missed message formats.
In our email mirror, we have a special format for missed
message emails that uses a 32-bit randomly generated token
that we put into redis that is then prefixed with "mm" for
a total of 34 characters.

We had a bug where we would mis-classify emails like
mmcfoo@example.com as being these system-generated emails
that were part of the redis setup.

It's actually a little unclear how the bug in the library
function would have manifested from the user's point of view,
but it was definitely buggy code, and it's possibly related in
a subtle way to an error report we got from a customer where
only one of their users, who happened to have a name like
mmcfoo, was having problems with the mirror.
2016-09-22 13:41:26 -07:00
Steve Howell 0b7cac04d4 email mirror: Extract is_mm_32_format(). 2016-09-22 13:41:26 -07:00
Tim Abbott 68c51912c9 extract_recipients: Add a type: ignore for problematic arguments.
We'll want to redo this function to be less crazy, but this is a quick
fix to be able to upgrade mypy.
2016-09-19 21:48:12 -07:00
Tim Abbott e488d4d6e8 log_event: Fix unicode handling of log writing. 2016-09-19 21:45:52 -07:00
Tim Abbott 939528303d upload: Add type: ignore for mimetypes.guess_type stubs issue. 2016-09-19 21:40:08 -07:00
Steve Howell 6b3f945a7e dead code: Remove linebreak(). 2016-09-19 18:25:03 -07:00
Steve Howell c2277afd06 Simplify bulk_get_streams().
We can always expect a realm here.
2016-09-19 18:25:03 -07:00
Steve Howell eb09dd217d Simplify get_stream_backend.
Assume that get_stream_backend() always receives a realm, not a
realm id.  We can be pretty confident of that now due to mypy.
2016-09-19 18:25:03 -07:00
Steve Howell 1dbc94bcd9 annotations: Require a Realm for get_stream(). 2016-09-19 18:25:02 -07:00
Steve Howell feb94a90aa tests: Add ModelTest class for miscellaneous model tests. 2016-09-19 18:25:02 -07:00
Steve Howell 7d257b03fc test: Add coverage for Recipient.__unicode__(). 2016-09-19 18:25:02 -07:00
Steve Howell e0d59d9386 dead code: Remove Stream.create(). 2016-09-19 18:25:02 -07:00
Steve Howell a67d17b9f7 tests: Add coverage for RealmFilter.__unicode__(). 2016-09-19 14:57:53 -07:00