Commit Graph

3249 Commits

Author SHA1 Message Date
Rishi Gupta 4b60e06991 realm creation flow: No longer create RealmAlias's during sign-up flow.
There are a number of situations in which we will want to automatically
create one, but we should be a bit thoughtful about when we do so.
2016-11-03 13:53:01 -07:00
Rishi Gupta c0f1b5d9f6 Change Realm defaults to Community defaults.
Previously, we set restrict_to_domain and invite_required differently
depending on whether we were setting up a community or a corporate
realm. Setting restrict_to_domain requires validation on the domain of the
user's email, which is messy in the web realm creation flow, since we
validate the user's email before knowing whether the user intends to set up
a corporate or community realm. The simplest solution is to have the realm
creation flow impose as few restrictions as possible (community defaults),
and then worry about restrict_to_domain etc. after the user is already in.

We set the test suite to explictly use the old defaults, since several of
the tests depend on the old defaults.

This commit adds a database migration.
2016-11-03 13:53:01 -07:00
Tim Abbott 08ce391611 register: Fix using subdomains logic when subdomains is disabled. 2016-11-03 13:51:50 -07:00
Tim Abbott fa6aafd0a5 test_signups: Fix test_completely_open_domain.
This test seems intended to verify registration in the case of a
unique completely open domain; but because of the mit.edu realm, it
instead tested that a logic bug in the non-subdomains case was
present.
2016-11-03 13:44:51 -07:00
Tim Abbott 9f86638ce5 test_signups: Cleanup open realm registration tests.
This removes some confusing comments and unnecessary use of subdomain
logic.
2016-11-03 13:44:43 -07:00
Tim Abbott 9b6c199538 tests: Fix need for Internet to run backend test suite.
A few places where we weren't using mock.patch to mock out Hesiod
(DNS) queries have crept in.
2016-11-03 13:41:54 -07:00
Tim Abbott 90f57c92d4 compute_mit_user_fullname: Fix result for Python 3.
DNS replies are bytes, and usernames are strings.
2016-11-03 13:40:43 -07:00
K.Kanakhin c89a2ffe2f Add raised exception output to development tornado running.
Replaces the default tornado ioloop tracback exception handler in
debug mode to add output for exceptions to the logging.

Fixes #2150.
2016-11-03 11:16:12 -07:00
Tomasz Kolek 9026f56713 Improve English grammar in bitbucket integration modify comment message.
Add `a` between verb and comment word i.e. updated a [comment] instead of
updated [comment].
2016-11-03 10:18:46 -07:00
Umair Khan 4502872e8e subdomain: Test user registration through Google OAuth. 2016-11-03 10:09:24 -07:00
Steve Howell 9b04ec7472 Send dicts for cross realm users to JS.
We now send dictionaries for cross-realm bots.  This led to the
following changes:

* Create get_cross_realm_dicts() in actions.py.
* Rename the page_params field to cross_realm_bots.
* Fix some back end tests.
* Add cross_realm_dict to people.js.
* Call people.add for cross-realm bots (if they are not already part of the realm).
* Remove hack to add in feedback@zulip.com on the client side.
* Add people.is_cross_realm_email() and use it in compose.js.
* Remove util.string_in_list_case_insensitive().
2016-11-03 09:51:23 -07:00
Steve Howell af6512e782 Add feedback bot to the test database. 2016-11-03 09:42:27 -07:00
Steve Howell 6324fe9ddf Rename get_cross_realm_users to get_cross_realm_emails. 2016-11-03 09:42:27 -07:00
Tim Abbott bfff9e1968 do_send_messages: Fix sending messages to feedback-bot.
This feature was broken in the message_to_dict refactoring.
2016-11-02 23:42:05 -07:00
Rishi Gupta 70c29d8f90 do_create_realm: Remove "(open realm)" indication in sign-up message.
In preparation for a change to do_create_realm where we will use the
database default for restricted_to_domain rather than computing it within
do_create_realm, and due to which do_create_realm will no longer know
whether we are creating an open realm or not.
2016-11-02 22:46:34 -07:00
Rishi Gupta db0e509422 do_create_realm: Replace domain argument with string_id.
Turns string_id into a required argument, and domain into an optional
argument.
2016-11-02 22:46:34 -07:00
Rishi Gupta c1bbd8d70d models.py: Add get_realm_by_string_id to assist with Realm.domain migration.
Step 0 of a two step process:

1. Replace all occurances of get_realm(domain) with
   get_realm_by_string_id(string_id)

2. Rename get_realm_by_string_id to get_realm.
2016-11-02 22:46:34 -07:00
Rishi Gupta 9ef8536cc6 models.Realm: Require Realm.string_id to be non-NULL.
Adds a database migration, adds a new string_id argument to the management
realm creation command, and adds a short name field to the web realm
creation form when REALMS_HAVE_SUBDOMAINS is False.
2016-11-02 22:46:34 -07:00
Rishi Gupta 64bcd71d6e models.Realm: Rename subdomain to string_id.
Does a database migration to rename Realm.subdomain to
Realm.string_id, and makes Realm.subdomain a property.  Eventually,
Realm.string_id will replace Realm.domain as the handle by which we
retrieve Realm objects.
2016-11-02 22:46:34 -07:00
Tomasz Kolek 4dd8cd9879 Replace "Commit" to commits sha in commits comment events in git integrations. 2016-11-02 22:19:51 -07:00
Tomasz Kolek ce5bb22ace Normalize comment commit event in github integration. 2016-11-02 22:19:51 -07:00
Tomasz Kolek 6ad4cce35c Add object number/id to mention about Issue, PR, etc. to all git integrations. 2016-11-02 22:19:51 -07:00
Tomasz Kolek 762b6b5817 Add on betwen comment action and noun in comments messages from git integrations.
Improve English grammar in comment messages in git integrations
by adding 'on' between comment action and noun (PR, Issue, Commit).
2016-11-02 22:19:51 -07:00
Tomasz Kolek cbd05eb0b2 Normalize bitbucket commit, PR and issues comment event's subject and content. 2016-11-02 22:19:51 -07:00
Tomasz Kolek ef6a1328ce Normalize gitlab snippet or commit comment event's subject and content. 2016-11-02 22:19:51 -07:00
Tomasz Kolek 4fc1488edc Normalize gitlab issue or MR comment event's subject and content. 2016-11-02 22:19:51 -07:00
Steve Howell 6659664e59 Simplify/fix cross-realm validation in recipient_for_emails().
We now simply exclude all cross-realm bots from the set of emails
under consideration, and then if the remaining emails are all in
the same realm, we're good.

This fix changes two behaviors:
* You can no longer send a PM to an ordinary user in another realm
  by piggy-backing a cross-realm bot on to the message.  (This was
  basically a bug, but it would never manifest under current
  configurations.)
* You will be able to send PMs to multiple cross-realm bots at once.
  (This was an arbitrary restriction.  We don't really care about this
  scenario much yet, and it fell out of the new implementation.)
2016-11-02 22:10:49 -07:00
Steve Howell 497b8e8bc4 Make tests explicit about cross-realm loophole.
We can currently send a PM to a user in another realm, as long
as we copy a cross-realm bot from the same realm.  This loophole
doesn't yet affect us in practice--all cross-realm bots are
generally configured for the "admin" realm like the old zulip.com--
but we should lock it down in a subsequent commit.
2016-11-02 22:10:49 -07:00
Steve Howell 7d50598454 Test that non-zulip.com users can't PM random zulip.com users. 2016-11-02 22:10:49 -07:00
Steve Howell 45f9a60bc0 Add test coverage for talking to two cross-realm bots. 2016-11-02 22:10:49 -07:00
Steve Howell 067090d93e Test that we can send PMs to non-zulip cross-realm bots. 2016-11-02 22:10:49 -07:00
Steve Howell 7724dba08b Explicitly use the "feedback" bot in test_realm_scenarios.
(I'm about to create a more abstract cross-realm bot, but I think
concretely referring to feedback bot is helpful here.)
2016-11-02 22:10:49 -07:00
Steve Howell 517f7f672f Clean comments/whitespace in test_realm_scenarios(). 2016-11-02 22:10:49 -07:00
Steve Howell 94f53543e6 Test that users can PM themselves in test_realm_scenarios.
(This is probably tested elsewhere too, but it's helpful in the
context of testing cross-realm stuff to assert basic rules like
this.)
2016-11-02 22:10:49 -07:00
Steve Howell 7a127148db Add assert_message_received() to test_realm_scenarios. 2016-11-02 22:10:49 -07:00
Steve Howell b9bf74d54e Reorganize sections of test_realm_scenarios().
This change groups the positive scenarios together
and likewise for the negative scenarios.
2016-11-02 22:10:49 -07:00
Steve Howell 49c64bd7cb Consolidate cross-realm message tests.
Having each condition in a separate test was confusing to read,
especially since the tests were doing inconsistent setup, sometimes
calling user2 the user from 2.example.com realm and other times
calling user2 the cross-bot realm, etc.
2016-11-02 22:10:49 -07:00
umkay 0e0a11bb5e Refactor GitHub authentication backend tests. 2016-11-02 21:06:17 -07:00
Umair Khan 0eff88ec2d Test registration of mirror dummy user.
Tweaked by tabbott to do the from_confirmation=1 call.
2016-11-02 18:17:29 -07:00
Umair Khan dd0c4ca0bc Test registration using POPULATE_PROFILE_VIA_LDAP. 2016-11-02 17:50:40 -07:00
Umair Khan f300c12535 Accept from_confirmation kwarg in submit_reg_form_for_user. 2016-11-02 17:50:40 -07:00
Umair Khan 99f4174894 Test special registation link for completely open realms. 2016-11-02 17:50:39 -07:00
Umair Khan 8ff2cd3fd7 Only allow proper url for registration.
This commit fixes a bug which occurs when a user tries to bypass the
correct registration url for the completely open domains.
2016-11-02 17:49:30 -07:00
Tim Abbott bf1a7a7f2d test_bugdown: Fix misplaced type annotation. 2016-11-01 20:40:22 -07:00
paxapy 1d1c41551b bugdown: Auto-linkify #StreamName. 2016-11-01 19:59:21 -07:00
paxapy d965ff16fd bugdown: refactor AutoLink to use VerbosePattern.
This makes it possible to reuse the verbose pattern logic for other
regexs as well.
2016-11-01 19:50:59 -07:00
Steve Howell 403490a95b annotations: Fix to_non_negative_int(). 2016-10-28 13:48:44 -07:00
Steve Howell 7111bb1744 Deprecate /json/streams.
Note that we still need the equivalent function in our
user-facing API, so there is not much code removal yet.
(Also, we will probably always keep this in our API,
as bot authors will usually just want a simple endpoint
here, whereas our client code gets page_params and events.)
2016-10-28 13:44:46 -07:00
Steve Howell 768a04d2fa Send user_ids for update events.
Here we handle a few flavors of realm_bot/update
and realm_user/update.  (Different events send different
attributes.)
2016-10-28 13:40:14 -07:00
Steve Howell 3fd7ab950a Send user_id to the client when removing bots/users.
This touches these events:

    realm_bot/remove
    realm_user/remove
2016-10-28 13:40:14 -07:00
Steve Howell 7883cecf28 Send user_id to the client in more places.
This commit touches:

    realm_bot/add
    realm_user/add
    page_params.bot_list
    page_params.people_list
2016-10-28 13:40:14 -07:00
Tim Abbott 1a88ca4da1 queue_processors: Fix feedback bot sender.
This fixes an issue where emails sent by feedback-bot had a forged
sender and thus sometimes had delivery issues.
2016-10-28 11:04:08 -07:00
Umair Khan 4cf7040100 subdomains: Use redirect_and_log_into_subdomain for registration.
This fixes a bug where the user who registered a new organization
wasn't correctly logged into that organization.
2016-10-28 11:04:08 -07:00
Umair Khan 2dabfc562c subdomains: Add tests for single domain OAuth2. 2016-10-28 11:04:08 -07:00
Umair Khan 26646abe8c Authenticate subdomains using single Google OAuth entry.
Previously, we used to create one Google OAuth callback url entry
per subdomain. This commit allows us to authenticate subdomain users
against a single Google OAuth callback url entry.
2016-10-28 11:04:08 -07:00
Umair Khan d139c9cb71 Add tests for maybe_send_to_registration function. 2016-10-28 09:27:56 -07:00
Umair Khan 8064ed1c0b Add test for ZulipRemoteUserBackend. 2016-10-28 09:27:56 -07:00
Umair Khan 839cab3434 Add tests for GitHubAuthBackend. 2016-10-28 09:27:56 -07:00
Umair Khan b6046bc77e Add tests for password_auth_enabled function. 2016-10-28 09:27:56 -07:00
Umair Khan d9e1975d2c Add ZulipAuthMixin tests. 2016-10-28 09:27:56 -07:00
Umair Khan 6dbe425675 Add Google OAuth2 backend tests. 2016-10-28 09:27:56 -07:00
Umair Khan 088d8eebad Add ZulipLDAPUserPopulator test. 2016-10-28 09:27:56 -07:00
Umair Khan 50422e775b Add LDAP tests. 2016-10-28 09:27:55 -07:00
K.Kanakhin 78b1b80987 caching: Add configuration class for post-migration cache flushing.
- To avoid redefining migrate manage command is added new application
  configuration class which emit post_migration signal. This signal
  require models module inside application and defined AppConfig
  Instance as signal sender.  Documentation here:
  https://docs.djangoproject.com/en/1.8/ref/signals/#post-migrate.
- Add AppConf subclass to __init__ zerver app file to make apllication
  load it by default.

Fixes #1084.
2016-10-27 23:26:34 -07:00
Tim Abbott 9b7a3f040c Remove now-unused /json/get_events endpoint. 2016-10-27 21:34:58 -07:00
Tim Abbott 86e933a4a1 process_queue: Suppress USING_RABBITMQ warnings in test suite. 2016-10-27 12:36:06 -07:00
Umair Khan 486774b28d Add tests for JWT based login. 2016-10-26 14:40:56 -07:00
Umair Khan 52acdd28bd Fix JWT code in the auth backend.
For some reason, we use 'load' function but it doesn't exist in the JWT
library code. This commit updates the code to use the correct interface
of the JWT library.

The signature verification is done by the decode function.
2016-10-26 14:40:56 -07:00
Arpith Siromoney c5510599ef Typing backend: use dict instead of user profile for sender in event.
Fixes #2110.
2016-10-26 13:12:18 -07:00
K.Kanakhin 39e0886361 retention-policy: Add tool to determine expired messages.
This is a first step towards implementing a message retention policy
feature.

- Add Realm model message_retention_days field to setup
  messages expired period for realm.
- Add migration.
- Add tool to get expired messages for each Realm.
- Add tests to cover tool for getting expired messages.
2016-10-25 15:38:08 -07:00
umkay 1d077d1a4f bugdown: Remove safe_mode dependencies.
Now that we have updated python-markdown, we remove the deprecated
safe_mode. We used safe_mode to escape raw html, so now instead we
pass in an EscapeHtml markdown extension to the markdown engine.

See https://pythonhosted.org/Markdown/release-2.6.html for details on
the deprecation.

Fixes: #2037 (also addresses the remaining piece of #2043).
2016-10-25 15:22:18 -07:00
umkay 5fe6e7c07c bugdown: Add tests for escaping raw html.
Adds additional tests in bugdown-data.json to check that we escape raw
html properly with script tags.

Many of these tests are based on the bleach "advanced" test suite.
2016-10-25 15:21:46 -07:00
Tim Abbott 7b9d1a7824 auth: Simplify REALMS_HAVE_SUBDOMAINS redirect code.
This logic was just a duplicate on realm.uri, except that in the case
that subdomains are disabled, it preserves the host in the request.
2016-10-25 13:55:17 -07:00
hackerkid b5816bf99e Don't use zulipdev.com when subdomain support is disabled. 2016-10-25 13:52:13 -07:00
Arpith Siromoney 01665d9c3c Extract get_recipient_user_profiles.
This creates a common function for extracting recipients, which both
do_send_messages and do_send_typing_notifications can use.
2016-10-25 13:37:01 -07:00
Tomasz Kolek 4790316b57 Add user setting option to always send push notifications.
Add option in user's settings for getting mobile push notifications
even if a Zulip browser is online.  Default is False.

Fixes: #1596.
2016-10-25 10:52:29 -07:00
Rishi Gupta b6da450b96 admin.js: Use RealmAlias instead of Realm to show the restricted domains.
Passes the allowed domains for a realm to the frontend, via
page_params.domains. Groundwork for allowing users to add and
remove domains via the admin setting page, rather than via the
realm_alias.py management command.
2016-10-25 10:07:20 -07:00
Rishi Gupta 537ee75761 Create RealmAlias entries for Realm.domain.
This is a preliminary step towards eliminating the realm.domain field
in favor of realm.subdomain.  Includes a database migration to create
these for existing realms.
2016-10-25 10:06:39 -07:00
Brock Whittaker fca61b2031 Add medium size avatars for use on the user's own settings page.
This adds a medium (500px) size avatar thumbnail, that can be
referenced as `{name}-medium.png`.  It is intended to be used on the
user's own settings page, though we may come up with other use cases
for high-resolution avatars in the future.

This will automatically generate and upload the medium avatar images
when a new avatar original is uploaded, and contains a migration
(contributed by Kirill Kanakhin) to ensure all pre-existing avatar
images have a medium avatar.

Note that this implementation does not provide an endpoint for
fetching the medium-size avatar for another user.

[substantially modified by tabbott]
2016-10-25 09:42:14 -07:00
Umair Khan 9b42b7fad3 Add tests for TestZulipRemoteUserBackend. 2016-10-25 08:13:35 -07:00
Umair Khan 9f37b9b386 Add tests for TestDevAuthBackend. 2016-10-25 17:57:33 +05:00
Tim Abbott e714ddbfc8 gather_subscriptions_helper: Fix mypy error. 2016-10-22 22:37:05 -07:00
Tim Abbott c59cdbb92d gather_subscriptions: Fix missing subscriber data in email_dict.
When we added data on never_subscribed streams to what
populate_subscribers is called on, we failed to add the corresponding
data on subscribers to email_dict, the mapping of user IDs to emails
for the subscribers.
2016-10-22 21:37:05 -07:00
Tim Abbott 14018353fc Fix never_subscribed streams being sent for Zephyr mirroring.
Because in the Zephyr world, stream names can be a secret, and also
Zephyr mirroring tends to involve many thousands of streams, we
shouldn't send this data.
2016-10-22 21:36:11 -07:00
Tim Abbott 893679baff Add migration to remove system avatar source.
Fixes the last commit having broken master.
2016-10-22 20:07:21 -07:00
Tim Abbott 5bea2f5e20 Remove unused AVATAR_FROM_SYSTEM code.
This is some of the code we'd need if we wanted to have Zulip generate
avatars for things.  Since it is so little useful code, and it's not
clear we will need this feature ever, we can remove this code to make
the codebase less confusing.  It'd be easy to dig this out of history
if we ever want it.

Fixes #2101.
2016-10-22 19:48:50 -07:00
Steve Howell 4d6af33bad Use make_stream() in test_subs.py. 2016-10-22 18:09:34 -07:00
Steve Howell 2a739318b9 Use make_stream() in test_signup.py. 2016-10-22 18:09:34 -07:00
Steve Howell 8410159af2 Use make_stream() in test_narrow.py. 2016-10-22 18:09:34 -07:00
Steve Howell b1ebb09c96 Use make_stream() in test_messages.py. 2016-10-22 18:09:34 -07:00
Steve Howell 0861421912 Use make_stream() in test_events.py. 2016-10-22 18:09:34 -07:00
Steve Howell 79b6f3d39d Create ZulipTestCase.make_stream(). 2016-10-22 18:09:34 -07:00
K.Kanakhin 16c3da6db5 tests: Add tests for SEND_MISSED_MESSAGE_EMAILS_AS_USER=False.
- Add tests for SEND_MISSED_MESSAGE_EMAILS_AS_USER is False (the
  default!).
- Reorganized test case code by removing repeated parts of code,
  improving code style and moving common parts to separate class
  methods.

Fixes #1697.
2016-10-21 08:47:24 -07:00
Umair Khan 8891731d7d is_template_database_current: Open file in write mode. 2016-10-21 08:35:30 -07:00
Umair Khan b894135fae Check postgres-init-dev-db in is_template_database_current. 2016-10-21 08:35:30 -07:00
Umair Khan 176825d335 Refactor is_template_database_current.
Now all the files whose hash is to be checked are passed as a list
argument.
2016-10-21 08:35:30 -07:00
Umair Khan bf4e0800e3 is_template_database_current: Create hashes in var/test_db_status. 2016-10-21 08:35:30 -07:00
Umair Khan 8ade9e2408 Add tests to check subdomains with auth decorators.
Fixes: #1870
2016-10-21 12:21:47 +05:00
Tomasz Kolek 6e1674aca5 Normalize github issue or PR comment event's subject and content. 2016-10-20 23:15:51 -07:00
Tomasz Kolek 56dd1a9b90 Normalize gitlab issue event's subject and content. 2016-10-20 23:15:30 -07:00