Commit Graph

1973 Commits

Author SHA1 Message Date
Vishnu Ks 43a6439b3b upload: Enforce per-realm quota. 2018-01-29 16:06:11 -08:00
Greg Price 55cf54c087 upload: Remove old per-user quota feature.
We'll replace this primarily with per-realm quotas (plus the simple
per-file limit of settings.MAX_FILE_UPLOAD_SIZE, 25 MiB by default).

We do want per-user quotas too, but they'll need some more management
apparatus around them so an admin has a practical way to set them
differently for different users.  And the error handling in this
existing code is rather confused.  Just clear this feature out
entirely for now; then we'll build the per-realm version more cleanly,
and then we can later add back per-realm quotas modelled after that.

The migration to actually remove the field is in a subsequent commit.

Based in part on work by Vishnu Ks (hackerkid).
2018-01-29 16:06:11 -08:00
Greg Price 0f99610478 signup: Small refactor to isolate actually sending mail.
This is a little cleaner in that the try/except blocks for
SMTPException are a lot narrower; and it'll facilitate an upcoming
change to sometimes skip sending mail.
2018-01-29 10:27:11 -08:00
Greg Price 4bdecb7150 create_realm: Small refactor to check key just once. 2018-01-24 14:34:30 -08:00
YJDave 15dd94035c org settings: Handle NEW_USER_BOT not being set.
If NEW_USER_BOT is not configured in organization, then disable setting
for selecting stream for new user notification.
2018-01-23 15:23:25 -05:00
Brock Whittaker 849d63dd1c settings: Change error for "no data" to something more friendly. 2018-01-23 14:38:59 -05:00
Brock Whittaker ffe3a6c127 settings: Remove password confirmation in modal.
This removes the requirement to confirm your new password.
It isn't necessary and can be fixed easily with an email
reset if messed up.
2018-01-23 14:38:59 -05:00
Robert Hönig 9a15c4e3ff Add bot services to page_params.
This is the first step for allowing users
to edit a bot's service entries, name the
outgoing webhook configuration entries. The
chosen data structures allow for a future
with multiple services per bot; right now,
only one service per bot is supported.
2018-01-23 07:29:00 -05:00
Robert Hönig 06fb868482 Send an event to bot owners when a bot is created. 2018-01-23 07:29:00 -05:00
Eeshan Garg 00d3f19c0a integrations/email: Render settings.EMAIL_GATEWAY_EXAMPLE correctly. 2018-01-22 18:05:20 -05:00
Aditya Bansal f272ea9087 scheduledmessages: Start using/expecting delivery_type as a param. 2018-01-19 11:33:11 -05:00
Aditya Bansal c770bdaa3a reminder_bot: Add infra for adding reminder bot to every realm. 2018-01-19 11:33:11 -05:00
Aditya Bansal f46d098558 schedulemessages: Add handle_deferred_message() to handle requests.
This is responsible for:
1.) Handling all the incoming requests at the
messages endpoint which have defer param set. This is similar to
send_message_backend apart from the fact that instead of really
sending a message it schedules one to be sent later on.
2.) Does some preliminary checks such as validating timestamp for
scheduling a message, prevent scheduling a message in past, ensure
correct format of message to be scheduled.
3.) Extracts time of scheduled delivery from message.
4.) Add tests for the newly introduced function.
5.) timezone: Add get_timezone() to obtain tz object from string.
This helps in obtaining a timezone (tz) object from a timezone
specified as a string. This string needs to be a pytz lib defined
timezone string which we use to specify local timezones of the
users.
2018-01-19 11:33:11 -05:00
Robert Hönig d1d8365a6b embedded bots: Add config data UI.
This adds UI fields in the bot settings for specifying
configuration values like API keys for a bot. The names
and placeholder values for each bot's config fields are
fetched from the bot's <bot>.conf template file in the
zulip_bots package. This also adds giphy and followup
as embedded bots.
2018-01-09 11:50:54 -05:00
Alena Volkova 9d1063d362 urls: Move the json/fetch_api_key endpoint to be an API-style route. 2018-01-08 13:15:52 -05:00
Eeshan Garg 479bc41ca5 api docs: Remove old and outdated endpoint docs. 2018-01-04 10:17:29 -05:00
Alena Volkova 45f0c76c44 settings: Limit the creation of generic bots.
This commit adds a setting to limit creation of generic bots
to admins for realms that want that restriction.  (Generic
bots, apart from being considered spammy on some realms,
have less locked down permissions than webhook bots).

Fixes #7066.
2018-01-02 18:12:22 -05:00
Aastha Gupta daf86eb664 settings: Add "text" option to emoji_set model.
We no longer have a special UI setting and model
field ("emoji_alt_code") for saying users want text-only
emojis.  We now instead make "text" be a fifth choice
for "emojiset".

Fixes #7406
2018-01-02 14:55:01 -05:00
Vishnu Ks 277d463d23 messages: Add option to limit users message history. 2018-01-02 13:45:01 -05:00
YJDave fa44d2ea69 settings: Remove autoscroll_forever setting.
Fixes #6845
2018-01-02 10:35:49 -05:00
rht 1324d9f000 mypy: Use Python 3 type syntax in zerver/views. 2017-12-30 07:34:05 -05:00
YJDave 29905ad5d6 create stream: Before sending notificaion, check if NOTIFICATION_BOT exists.
There might be case that NOTIFICATION_BOT is none, so before sending stream
announce notification, check first if settings.NOTIFICATION_BOT is not none.
2017-12-29 08:59:55 -05:00
Kiy4h 6fa51be822 mypy: Use Python 3 type syntax in zerver/views/push_notifications.py. 2017-12-28 07:26:44 -05:00
Kiy4h 64413e3b91 mypy: Use Python 3 type syntax in zerver/views/email_mirror.py. 2017-12-28 07:26:44 -05:00
Kiy4h 238cd2b132 mypy: Use Python 3 type syntax in zerver/views/alert_words.py. 2017-12-28 07:26:44 -05:00
Xavier Cooney 36b58ff6fc mypy: Use Python 3 type syntax in 'views/users.py'. 2017-12-26 08:34:38 -05:00
nyan-salmon 0fd0332a2c mypy: Use Python 3 type syntax in zerver/views/custom_profile_fields.py. 2017-12-26 08:34:06 -05:00
Viraat Chandra 0494902e6d mypy: Use Python 3 syntax for typing in `zerver/views/auth.py`. 2017-12-26 08:31:43 -05:00
Xavier Cooney 61704dc20b mypy: Use Python 3 type syntax in views/pointer.py. 2017-12-26 08:29:30 -05:00
Callum Fraser b0c5eb5482 mypy: Use Python 3 type syntax in zerver/views/typing.py. 2017-12-18 11:32:52 -05:00
Greg Price 17a6632c43 zerver/views: Revert to Python 2 typing syntax for now (storage, streams).
This reverts commit 620b2cd6e.

Contributors setting up a new development environment were getting
errors like this:
```
++ dirname tools/do-destroy-rebuild-database
[...]
+ ./manage.py purge_queue --all
Traceback (most recent call last):
[...]
  File "/home/zulipdev/zulip/zproject/legacy_urls.py", line 3, in <module>
    import zerver.views.streams
  File "/home/zulipdev/zulip/zerver/views/streams.py", line 187, in <module>
    method_kwarg_pairs: List[FuncKwargPair]) -> HttpResponse:
  File "/usr/lib/python3.5/typing.py", line 1025, in __getitem__
    tvars = _type_vars(params)
[...]
  File "/usr/lib/python3.5/typing.py", line 277, in _get_type_vars
    for t in types:
TypeError: 'ellipsis' object is not iterable
```

The issue appears to be that we're using the `typing` module from the
3.5 stdlib, rather than the `typing=3.6.2` in our requirements files,
and that doesn't understand the `Callable[..., HttpResponse]` that
appears in the definition of `FuncKwargPair`.

Revert for now to get provision working again; at least one person
reports that reverting this sufficed.  We'll need to do more testing
before putting this change back in.
2017-12-13 10:20:11 -08:00
rht 620b2cd6e2 zerver/views: Use Python 3 syntax for typing (storage, streams). 2017-12-12 17:40:50 -08:00
Rhea Parekh 8ed7f73199 IRC mirror: No longer require IRC stream names to start with #. 2017-12-12 10:04:31 -06:00
Xavier Cooney 7887857c81 mypy: Use Python 3 type syntax in user_settings.py 2017-12-11 20:33:00 -08:00
picapi_ 2adeaab6ab mypy: Use Python 3 type syntax in zerver/views/invite.py. 2017-12-11 20:30:19 -08:00
Shreyansh Dwivedi 47fcb27e39 invitations: Remove custom_body.
Fixes #7672
2017-12-11 19:23:54 -08:00
Callum Fraser 00178d114b mypy: Use Python 3 type syntax in zerver/views/streams.py.
[greg: small formatting tweaks]
2017-12-11 18:43:24 -08:00
Rishi Gupta d9ce33dcf0 registration: Clarify realm_creation conditionals in accounts_register.
No change in behavior.
2017-12-11 18:36:14 -08:00
Rishi Gupta e545a681fc registration: Use standard pattern for returning link_does_not_exist.
Eventually this check for the realm will be done in get_object_from_key
itself. Rewriting this to fit the pattern in get_object_from_key.

No change to behavior.
2017-12-11 18:36:14 -08:00
Rishi Gupta dbe3576706 registration: Enforce realm is None only if realm_creation.
Commit d4ee3023 and its parent have the history behind this code.

Since d4ee3023^, all new PreregistrationUser objects, except those for
realm creation, have a non-None `realm`.  Since d4ee3023, any legacy
PreregistrationUsers, with a `realm` of None despite not being for
realm creation, are treated as expired.  Now, we ignore them
completely, and remove any that exist from the database.

The user-visible effect is to change the error message for
registration (or invitation) links created before d4ee3023^ to be
"link does not exist", rather than "link expired".

This change will at most affect users upgrading straight from 1.7 or
earlier to 1.8 (rather than from 1.7.1), but I think that's not much
of a concern (such installations are probably long-running
installations, without many live registration or invitation links).

[greg: tweaked commit message]
2017-12-11 18:36:14 -08:00
Arseny Chekanov e5c05f128a mypy: Use Python 3 type syntax in zerver/views/realm_filters.py. 2017-12-09 16:50:12 -08:00
Xavier Cooney aa37b50d1e mypy: User Python 3 syntax for typing in realm.py. 2017-12-08 18:34:26 -08:00
Xavier Cooney 63e793a8d1 mypy: Use Python 3 syntax for typing in views/tutorial.py. 2017-12-08 18:34:26 -08:00
Rishi Gupta 100167fcf8 invitations: Change error messages for resend/revoke views.
We don't have a concept of invitation ID on the frontend (or backend).
2017-12-06 20:35:50 -08:00
Rishi Gupta cb5eb47470 invitations: Move resend/revoke error handling out of actions.py. 2017-12-06 20:35:50 -08:00
picapi_ e2c14724d9 mypy: Use Python 3 type syntax in zerver/views/messages.py. 2017-12-03 13:48:18 -08:00
Alena Volkova 3be8e95d13 settings: Send bot types from the backend instead of hardcoding them. 2017-11-30 15:28:53 -08:00
Rishi Gupta 7c9694077e confirmation: Move check_prereg_key_and_redirect to registration.py. 2017-11-29 22:18:05 -08:00
Rishi Gupta 331a9bee6b confirmation: Add Confirmation.REALM_CREATION.
Does not change any behavior. Groundwork for adding a realm field to
Confirmation.
2017-11-29 22:03:09 -08:00
Sarah ba70b542d3 notification settings: Add stream email notifications setting.
Adds a field to UserProfile for enabling emails about stream
messages and a field to Subscriptions about email
notification preferences.
2017-11-28 17:51:18 -08:00
Greg Price 0ec2a9d259 auth: Try switching to register even if user exists on another realm.
For example, this means that if a user already has an account on one
realm and they try to make an account on another by hitting "Sign in
with Google" (rather than following the little "Register" link to a
"Sign up with Google" button instead), they'll get to make an account
instead of getting an error.

Until very recently, if the user existed on another realm, any attempt
to register with that email address had to fail in the end, so this
logic gave the user a useful error message early.  We introduced it in
c23aaa178 "GitHub: Show error on login page for wrong subdomain"
back in 2016-10 for that purpose.  No longer!  We now support reusing
an email on multiple realms, so we let the user proceed instead.

This function's interface is kind of confusing, but I believe when its
callers use it properly, `invalid_subdomain` should only ever be true
when `user_profile` is None -- in which case the revised
`invalid_subdomain` condition in this commit can never actually fire,
and the `invalid_subdomain` parameter no longer has any effect.  (At
least some unit tests call this function improperly in that respect.)
I've kept this commit to a minimal change, but it would be a good
followup to go through the call sites, verify that, eliminate the use
of `invalid_subdomain`, then remove it from the function entirely.
2017-11-28 16:38:41 -08:00
Tim Abbott 586e18b237 auth: Allow accounts with the same email in different realms.
[Modified by greg to (1) keep `USERNAME_FIELD = 'email'`,
(2) silence the corresponding system check, and (3) ban
reusing a system bot's email address, just like we do in
realm creation.]
2017-11-28 16:23:10 -08:00
Tim Abbott e6f460f511 auth: Replace user_email_is_unique validator.
As we migrate to allow reuse of the same email with multiple realms,
we need to replace the old "no email reuse" validators.  Because
stealing the email for a system bot would be problematic, we still ban
doing so.

This commit only affects the realm creation logic, not registering an
account in an existing realm.
2017-11-28 16:23:10 -08:00
Greg Price 5f38b374b6 register: Activate existing mirror dummy only if on same realm.
The one thing this bit of logic is used for is to decide whether
there's an existing user which is a mirror dummy that we should
activate.  This change causes us to ignore such an existing user if
it's on some other realm, and go straight into `do_create_user`.
2017-11-28 16:23:10 -08:00
Tim Abbott 4f5a5a8547 auth: Extract add_dev_login_context.
This simple refactor cleans up the code for the dev_auth_enabled code
path to be a bit less deeply in the login_page() logic.
2017-11-28 15:17:33 -08:00
derAnfaenger c8a5ae753c embedded bots: Consistently use 'storage' instead of 'state.' 2017-11-27 21:05:34 -08:00
rht e538f4dd44 zerver/views: Use Python 3 syntax for typing.
Edited by tabbott to remove state.py and streams.py, because of
problems with the original PR's changes, and wrap some long lines.
2017-11-27 17:10:39 -08:00
Tim Abbott bd8196c3a5 lint: Fix too-long line in auth.py. 2017-11-27 15:28:46 -08:00
Vishnu Ks d4ee3023b2 registration: Require an explicit realm on PreregistrationUser.
This completes the last commit's work to fix CVE-2017-0910, applying
to any invite links already created before the fix was deployed.  With
this change, all new-user registrations must match an explicit realm
in the PreregistrationUser row, except when creating a new realm.

[greg: rewrote commit message]
2017-11-27 14:59:34 -08:00
Vishnu Ks 985768b2fd registration: Check realm against PreregistrationUser realm.
We would allow a user with a valid invitation for one realm to use it
on a different realm instead.  On a server with multiple realms, an
authorized user of one realm could use this (by sending invites to
other email addresses they control) to create accounts on other
realms. (CVE-2017-0910)

With this commit, when sending an invitation, we record the inviting
user's realm on the PreregistrationUser row; and when registering a
user, we check that the PregistrationUser realm matches the realm the
user is trying to register on.  This resolves CVE-2017-0910 for
newly-sent invitations; the next commit completes the fix.

[greg: rewrote commit message]
2017-11-27 14:58:26 -08:00
Tim Abbott 2210f627a5 signup: Switch active mirror-dummy users to an AssertionError.
Previously, this was a ValidationError, but that doesn't really make
sense, since this condition reflects an actual bug in the code.

Because this happened to be our only test coverage the ValidationError
catch on line 84 of registration.py, we add nocoverage there for now.
2017-11-26 16:13:45 -08:00
Vishnu Ks 9f225360dc register: Make /register redirect to find accounts in root domain without realm.
If there is no realm in the root domain don't show
registration form. Instead just redirect to find
accounts page.
2017-11-26 15:32:17 -08:00
Robert Hönig 0e0a8a2b14 queue processor tests: Call consume by default.
This significantly improves the API for queue_json_publish to not be
overly focused on what the behavior of this function should be in our
unit tests.
2017-11-26 11:45:34 -08:00
rht e169bb0954 zerver: Remove simplejson dependency.
Modified by tabbott to put the third-party code in a new file.

Fixes #6970.
2017-11-21 21:51:48 -08:00
derAnfaenger e526d0c144 embedded bots: Add views to access state. 2017-11-21 21:10:39 -08:00
Eeshan Garg 73d0f1db81 lib/integrations: Remove Integration.doc_context.
An Integration object doesn't need access to the context dict used
to render its doc.md, since the context dict is just passed directly to
render_markdown_path.
2017-11-21 20:39:30 -08:00
Eeshan Garg 600ee90a38 integrations: Add add_context_for_single_integration function.
Previously, when rendering a single integration, we tacked on the
following information to the context dict that was redundant:

* An OrderedDict containing all of the Integration objects for
  all integrations.
* An OrderedDict containing all of the integration categories.

The context dict for rendering a particular integration doc would
contain 4 OrderedDicts, 2 for categories, 2 for Integration objects
because of how many times add_integrations_context had been called.

This was very wasteful, since an Integration object doesn't need
to access any other Integration object (or itself for that matter)
to render its documentation. This commit adds a function that
allows us to only pass in the context values that are necessary.
2017-11-21 20:39:30 -08:00
Tim Abbott 3bfb19b5f3 Convert EmailAuthBackend and LDAPAuthBackend to accept a realm. 2017-11-21 18:23:50 -08:00
Tim Abbott 1b95b098dd auth: Clarify comments explaining the GoogleMobileOauth2Backend. 2017-11-21 18:23:50 -08:00
Tim Abbott a7d51127fb auth: Convert GoogleMobileOAuth2Backend to accept a realm object. 2017-11-21 18:23:49 -08:00
Tim Abbott 07bc31f818 auth: Convert DevAuthBackend to accept a realm object. 2017-11-21 18:23:49 -08:00
Tim Abbott 4968631d1b auth: Convert DevAuthBackend to use a unique argument pattern.
This helps ensure that we won't accidentally activate this backend on
other code paths.
2017-11-21 18:23:49 -08:00
Tim Abbott 387c9109ec auth: Convert RemoteUserBackend to accept a realm object. 2017-11-21 18:23:49 -08:00
Tim Abbott f17974ab32 DummyAuthBackend: Require being passed a realm object.
We should now always know the realm in our auth code paths.
2017-11-21 18:22:37 -08:00
Vishnu Ks 34689370cd settings: Create UI and backend for setting signups stream. 2017-11-21 17:39:51 -08:00
Rishi Gupta 42652713c8 onboarding: Refactor away setup_initial_private_stream.
It is a confusing name, now that the stream creation part has been taken
away.
2017-11-21 17:39:50 -08:00
Tim Abbott 29408d24ff emoji: Rework reactions validation to have a single function.
This feels more readable to me, and also identified a couple cases
where we were missing test coverage.
2017-11-20 15:27:21 -08:00
Tim Abbott 2d4c7b771a reactions: Move legacy endpoint code further down in file.
This helps ensure that new contributors working on the reactions code
read the stuff we use now first.
2017-11-20 15:27:21 -08:00
Tim Abbott 80a2a36dd1 reactions: Rename legacy reactions for greater clarity. 2017-11-20 15:27:21 -08:00
Tim Abbott 25ddba99f6 lint: Ban general use of user_profile.save().
This often can cause minor caching problems.

Obviously, it'd be better if we had access to the AST and thus could
do this rule for UserProfile objects in general.
2017-11-20 10:57:08 -08:00
Vishnu Ks 432c9d3449 email_log: Reset the value of email after email change process.
This commit actually needs only user_profile.save() line
but it's more readable with both the lines.
2017-11-20 10:45:16 -08:00
Tim Abbott c8edbae21c password reset: Fix error message for invalid realm.
This is a lot cleaner than the previous model.

Basically rewritten by Vishnu Ks to actually work :).
2017-11-20 10:34:55 -08:00
Eeshan Garg 9138d3b8d7 integrations: Move integration-specific context to integration_doc.
Instead of populating the context dict with integration-specific
information in render_markdown_path, we now do that in
zerver.views.integrations.integration_doc instead.

Fixes #7401.

Tweaked by tabbott to use cast to handle the typing issues here.
2017-11-19 20:48:46 -08:00
Tim Abbott 18369cb5ad JWT: Fetch the subdomain earlier in the auth code path. 2017-11-17 17:33:28 -08:00
Harshit Bansal d9c2f613e3 api: Add new endpoint for reactions.
This endpoint will allow us to add/delete emoji reactions whose emoji
got renamed during various emoji infra changes. This was also a
required change for realm emoji migration.

This commit was tweaked significantly by tabbott for greater clarity
(with no changes to the actual logic).
2017-11-16 20:52:15 -08:00
Sampriti Panda a696141a25 search: Add more extensive logging for Unicode errors in highlight_string.
Tweaked by tabbott to provide finer-grained logging.
2017-11-16 15:08:16 -08:00
Tim Abbott 054952a44a docs: Update links from codebase to point to ReadTheDocs. 2017-11-16 10:53:49 -08:00
Umair Khan cc76f7deac user-groups: Create check_delete_user_group. 2017-11-15 22:35:43 -08:00
Umair Khan 66179ea295 bulk_add_members_to_user_group: Pass user group.
Previously we passed user group id. If we pass user group, send event is
easier.
2017-11-15 22:35:23 -08:00
Tim Abbott c7a975e4df users: Move check_change_full_name to actions.py.
This avoids an import loop in the next commit, and better matches our
usual code structure.
2017-11-15 17:39:09 -08:00
Tim Abbott 03b4a0240c user_settings: Fix mypy annotations.
While we're at it, convert this to use the Python 3 style, which isn't
prone to this sort of bug.
2017-11-15 17:38:19 -08:00
Brock Whittaker f9f0f356be settings: Add backend storage and interface for night mode.
This allows the night mode setting to be stored in the backend.
2017-11-15 16:36:06 -08:00
Tim Abbott 85844107fd integrations: Remove code for legacy Hubot lozenges.
We no longer display the only model anyway.
2017-11-15 14:04:19 -08:00
rht bafba95eb7 Text-wrap the remaining long lines exceeding 110. 2017-11-15 10:58:03 -08:00
Tim Abbott 5aa6751e7a lint: Fix long lines in newly added features. 2017-11-15 10:58:03 -08:00
Vishnu Ks 36f29764cb emails: Add option to forward mails send in dev env to external email.
Fixes #7085.
2017-11-14 15:27:29 -08:00
Vishnu Ks d2555e8769 api: Add support to change name of default stream groups. 2017-11-14 14:41:42 -08:00
Vishnu Ks 2af249dd5d api: Add description attribute to default stream group. 2017-11-14 14:41:42 -08:00
Vishnu Ks f45ba7de93 api: Pass group id instead of name to default stream group api. 2017-11-14 14:41:42 -08:00
rht a93b7c40a7 zerver: Text-wrap long lines exceeding 110. 2017-11-10 16:29:32 -08:00
Vishnu Ks f44b60a150 Implement API for default stream groups. 2017-11-10 16:05:36 -08:00
Umair Khan b1603d289c user-groups: Add update members API endpoint.
Significantly modified by tabbott to fix some bugs.
2017-11-09 17:35:37 -08:00
Umair Khan a1b894a786 user-groups: Add delete API endpoint.
Significantly modified by tabbott to fix some bugs.
2017-11-09 17:35:30 -08:00
Umair Khan eeaf0f1742 user-groups: Add edit API endpoint.
Significantly modified by tabbott to fix some bugs.
2017-11-09 17:35:22 -08:00
Umair Khan 1bbe84af49 user-groups: Add create API endpoint.
Significantly modified by tabbott for better security structure.
2017-11-09 17:26:14 -08:00
Steve Howell c38996ddde Remove unused force_text import. 2017-11-09 10:32:14 -08:00
Steve Howell 786e90d774 Removed unused imports of force_bytes. 2017-11-09 10:32:14 -08:00
YJDave de67bf81fd org settings: Allow users to delete their message.
Fixes #7049.
2017-11-09 10:20:34 -08:00
rht 88a828dd0c Remove six.moves.urllib usage. 2017-11-09 10:00:00 -08:00
derAnfaenger 19bc55aa45 Fix various typos.
The typos and their corrections were found with the
aid of https://github.com/lucasdemarchi/codespell.
2017-11-09 16:26:38 +01:00
Tim Abbott a8c735e1d2 /api/: Fix some headings. 2017-11-08 13:08:23 -08:00
Eeshan Garg 7c113f0447 api-docs: Render api_url in /api docs.
In templates/zerver/api/main.html, since the current context isn't
passed to render_markdown_path when rendering an article,
render_markdown_path doesn't have the context to render values such
as api_url. This commit makes sure that it does by passing a dict
called api_uri_context to render_markdown_path when rendering an
article.
2017-11-08 01:29:49 -03:30
Steve Howell 4108797218 Eliminate the parse_usermessage_flags interface.
This commit puts the guts of parse_usermessage_flags into
UserMessage.flags_list_for_flags, since it was slightly faster
than the old implementation and produced the same results.
(Both algorithms were super fast, actually.)

And then all callers use the model method now.
2017-11-07 17:48:27 -08:00
Steve Howell ae0b27a7ed Extract messages_for_ids. 2017-11-07 17:48:27 -08:00
Steve Howell 70f2cd2860 De-duplicate is_search logic in get_messages_backend().
The logic to set search_fields was essentially the same for both
sides of the include_history conditional.

Now we have just one code block that sets search_fields, and we
can quickly short-circuit the loop when is_search is False.
2017-11-07 17:48:27 -08:00
Steve Howell c62588c359 minor: Use dict comprehension in get_messages_backend(). 2017-11-07 17:48:27 -08:00
Tim Abbott a8a159c5d2 mypy: Fix variable reuse in streams.py. 2017-11-07 17:48:27 -08:00
rht 296835351d zerver/views: Text-wrap long lines exceeding 110.
Tweaked by tabbott to fix various minor issues.
2017-11-07 17:24:09 -08:00
Rishi Gupta 15b3a8e4ff email change: Use confirmation object user to determine changeability.
Seems like the more logical check. Also, the previous code makes it feel
like there is a potential vulnerability where one could get an email change
object in a realm where email changes are disabled, and then open that link
while logged in to a different realm.

While we're at it, remove the unnecessary check that the user is
logged in when clicking the confirmation link; that creates
unnecessary trouble for users who use multiple browsers.
2017-11-07 16:26:27 -08:00
Rishi Gupta 4fc8aa2b53 email change: Refactor confirmation path for readability.
Removes an assert, which at this point is there just for readability, since
the second argument to
get_object_from_key(confirmation_key, Confirmation.EMAIL_CHANGE)
ensures that the returned object is of the correct type.
2017-11-07 16:02:49 -08:00
Rishi Gupta 6fbb7e8046 email change: Remove confirmation_key.lower() from confirm_email_change.
Not sure why it is here, and it adds confusion.
2017-11-07 16:02:49 -08:00
Rishi Gupta 1823d46236 unsubscribe: Use a custom confirmation error page.
Reverts a bit of 6e02ce8.
2017-11-07 16:02:49 -08:00
rht 14671fc7b9 refactor: Remove six.moves.zip_longest import. 2017-11-07 10:51:44 -08:00
rht 80a8d4f9f3 refactor: Remove six.moves.map import. 2017-11-07 10:46:42 -08:00
rht 549a26860f refactor: Remove six.moves.range import. 2017-11-07 10:46:42 -08:00
rht ec5120e807 refactor: Remove six.moves.zip import. 2017-11-07 10:46:42 -08:00
rht 19bd335cbb Change urllib import to be Python 3-specific. 2017-11-07 10:46:42 -08:00
Steve Howell 8450f0ec69 Register client_gravar=True for the webapp. 2017-11-07 10:42:18 -08:00
Steve Howell 2bbfda041a Support client_gravatar field for event registration.
This commit allows clients to register client_gravatar=True, and
then we recognize that flag for message events.  If the flag is
True, we will not calculate gravatar URLs and let the clients do
it themselves.  (Clients can calculate gravatar URLs based on
emails with just a little bit of code.)
2017-11-07 10:42:17 -08:00
rht 8990b1046d zerver: Remove inheritance from object. 2017-11-06 08:53:48 -08:00
neiljp (Neil Pilgrim) 42f5eea61f mypy: Improve typing of request.pyi (REQ).
This expands request.pyi to type-check the arguments passed into REQ.

Tweaked by tabbott to fix the report.py annotations.
2017-11-04 20:26:03 -07:00
neiljp (Neil Pilgrim) 186181c2a0 mypy: Specify type parameters in convert_term in narrow_parameter. 2017-11-04 19:47:45 -07:00
neiljp (Neil Pilgrim) da97de3370 mypy: Use Dict[Any,Any] for update_realm parameter. 2017-11-04 19:47:45 -07:00
Greg Price 68b0a419ec decorator: Cut a bunch of dead imports of two view decorators.
Saw these when grepping for these two decorators; they're actually
more numerous than the surviving use sites are.  Cut out the noise.
2017-11-04 19:27:00 -07:00
Greg Price 6b9d294873 auth: Set user_activity `query` nicely for several auth views.
This gets used when we call `process_client`, which we generally do at
some kind of login; and in particular, we do in the shared auth
codepath `login_or_register_remote_user`.  Add a decorator to make it
easy, and use it on the various views that wind up there.

In particular, this ensures that the `query` is some reasonable
constant corresponding to the view, as intended.  When not set, we
fall back in `update_user_activity` on the URL path, but in particular
for `log_into_subdomain` that can now contain a bunch of
request-specific data, which makes it (a) not aggregate properly, and
(b) not even fit in the `CHARACTER VARYING(50)` database field we've
allotted it.
2017-11-04 19:27:00 -07:00
Rishi Gupta 5d04e29ac2 confirmation: Add comment for registration.confirmation_key.
I remember being really confused by this function in the past, and I finally
figured it out. It should be removed, and the dev_url added by
00-realm-creation should call a function that just gets the confirmation_key
from outbox like all of the backend tests, but until then this comment
should help.
2017-11-03 15:33:16 -07:00
Rishi Gupta 6e02ce8344 unsubscribe: Use get_object_from_key for confirmation.
This is the only one of the confirmation pathways that wasn't using this
idiom, I think.
2017-11-03 15:33:16 -07:00
Rishi Gupta fdbe36644e confirmation: Add confirmation_type to get_object_from_key.
This change:

* Prevents weird potential attacks like taking a valid confirmation link
  (say an unsubscribe link), and putting it into the URL of a multiuse
  invite link. I don't know of any such attacks one could do right now, but
  reasoning about it is complicated.

* Makes the code easier to read, and in the case of confirmation/views.py,
  exposes something that needed refactoring anyway (USER_REGISTRATION and
  INVITATION should have different endpoints, and both of those endpoints
  should be in zerver/views/registration, not this file).
2017-11-03 15:33:16 -07:00
Harshit Bansal 44bff43c86 actions.py: Rename `do_remove_reaction()` to `do_remove_reaction_legacy()`. 2017-10-31 15:42:35 -07:00
Harshit Bansal 233595b2f7 actions.py: Rename `do_add_reaction()` to `do_add_reaction_legacy()`. 2017-10-31 15:42:35 -07:00
Steve Howell 9689168bf6 Simplify highlight_string logic.
This change avoids some complicated byte arithmetic by
extracting some local variables and being more explicit
about using bytes vs. using strings.
2017-10-31 12:45:58 -07:00
Brock Whittaker 425738fa24 landing-page: Extract /team/ from /about/.
This extracts and creates a /team/ page using the existing
'team.html' template and applying the why-page styling to it.
2017-10-31 11:37:42 -07:00
Sampriti Panda ee61e1be2c search: Fix misaligned highlights due to unicode characters.
tsearch_extras returns search offsets in bytes but our highlight
function treated them as character offsets.  Added a check to subtract
extra bytes if the tsearch search backend is being used.

Fixes #4084.
Fixes #7021.
2017-10-31 09:06:46 -07:00
Greg Price 22e5110789 subdomains: Dedupe out the last copy of Realm.host_for_subdomain.
This makes the affected code somewhat easier to read, and paves the
way for changing this logic to add support for alias domains.
2017-10-30 18:29:29 -07:00
Greg Price eb55a3a1ba template context: Give better names to the URLs for the API.
The "subdomain" label is redundant, to the extent it's even
accurate -- this is really just the URL we want to display,
which may or may not involve a subdomain.  Similarly "external".

The former `external_api_path_subdomain` was never a path -- it's a
host, followed by a path, which together form a scheme-relative URL.
I'm not quite convinced that value is actually the right thing in
2 of the 3 places we use it, but fixing that can start by giving an
accurate name to the thing we have.
2017-10-30 18:29:29 -07:00
Greg Price 7366a4ca7a settings: Eliminate the disused EXTERNAL_API_PATH.
This setting isn't documented at all, and I believe nobody has used it
since the end of api.zulip.com in 2016.  So we get to complete the
cleanup of this logic.
2017-10-30 18:29:29 -07:00
rht c4fcff7178 refactor: Replace super(.*self) with Python 3-specific super().
We change all the instances except for the `test_helpers.py`
TimeTrackingCursor monkey-patching, which actually needs to specify
the base class.
2017-10-30 14:30:25 -07:00
Brock Whittaker 6933d51c0f views/integrations: Change non-generic HelpView to MarkdownDirectoryView.
The HelpView class will render a directory as markdown with an index HTML
page. This however can also be used for other generics and applied to
the API pages as well, so change the class to a generic class and
specify the path templates and names.

Tweaked by tabbott and Eeshan Garg.
2017-10-29 18:15:36 -07:00
Steve Howell 8b012c6210 Extract get_personal_recipient(). 2017-10-28 17:57:39 -07:00
Steve Howell a28841e8aa Extract get_stream_recipient().
Do you call get_recipient(Recipient.STREAM, stream_id) or
get_recipient(stream_id, Recipient.STREAM)?  I could never
remember, and it was not very type safe, since both parameters
are integers.
2017-10-28 17:57:39 -07:00
Steve Howell d132c30c24 Remove `active` parameter from `do_create_user()`.
Almost all callers to do_create_user were trying to
create active users, except for one test.  The
active=False codepath was kind of broken (things
like sending welcome messages had sort of undefined
behavior there), so instead of trying to maintain it,
we just update the one test (`test_people`) to flip the
`is_active` flag manually.

Fixes #7197
2017-10-28 10:32:01 -07:00
Tim Abbott 5b5dffebe7 get_messages_backend: Tweak how search fields are added.
This is a no-op, but makes mypy happier, since previously it wanted to
know the type of the {} we were passing to `.update()`.
2017-10-28 09:58:51 -07:00
Tim Abbott 80a9253f89 attachments: Remove unnecessary use of REQ.
It didn't do anything, because that third argument is actually a
positional argument coming from the URL parsing logic.
2017-10-27 15:18:44 -07:00
Tim Abbott 5467296f60 decorators: Use human_users_only more aggressively. 2017-10-27 15:16:13 -07:00
Tim Abbott 1cd017288d views: Fix imports of REQ/has_request_variables from the wrong place.
These were never in zerver/decorator.py, and so it makes sense to
import them zerver/lib/request.py, mostly for ease of finding things.
2017-10-27 15:07:31 -07:00
Greg Price d9cb606804 oauth: Find a better name for redirect_to_main_site.
This name was way too broad for the rather specific logic
in the actual function.
2017-10-27 14:42:24 -07:00
Greg Price ad551427ed auth: Factor out some uses of EXTERNAL_HOST with ROOT_DOMAIN_URI.
Apart from being less verbose, this makes it more manifest (on e.g. grep)
that we aren't using EXTERNAL_HOST here to construct subdomains.
2017-10-27 14:42:24 -07:00
Greg Price 318682fd52 auth: Use URL rather than cookie to pass signed data cross-domain.
The cookie mechanism only works when passing the login token to a
subdomain.  URLs work across domains, which is why they're the
standard transport for SSO on the web.  Switch to URLs.

Tweaked by tabbott to add a test for an expired token.
2017-10-27 14:42:04 -07:00
Greg Price fad3d56810 views: Move some login code from `registration` to `auth`.
Most of these have more to do with authentication in general than with
registering a new account.  `create_preregistration_user` could go
either way; we move it to `auth` so we can make the imports go only in
one direction.
2017-10-27 14:28:38 -07:00
Henrik Pettersson 09cd47c6ad Add UI for viewing and cancelling open Zulip invitations.
Lets administrators view a list of open(unconfirmed) invitations and
resend or revoke a chosen invitation.

There are a few changes that we can expect for the future:

  * It is currently possible to invite an email that you have already
    invited, it might make sense to change this behavior.

  * Resend currently sends an invite reminder instead of resending the
    original invite, this is because 'custom_body' was not stored when
    the first invite was sent.

Tweaked in various minor ways, primarily in the backend, by tabbott,
mostly for style consistency with the rest of the codebase.

Fixes: #1180.
2017-10-27 13:07:43 -07:00
Tim Abbott 237e3cdca1 registration: Clarify comment on do_activate_user code path. 2017-10-27 11:22:19 -07:00
Vishnu Ks 8c68a167fe backend: Allow Administrators to invite new users as admins.
Tweaked by tabbott to have the field before the invitation is
completed be called invite_as_admins, not invited_as_admins, for
readability.

Fixes #6834.
2017-10-27 11:19:38 -07:00
Steve Howell 65b6676844 minor: Remove unused import (internal_send_message). 2017-10-27 10:48:11 -07:00
Steve Howell 655f37a34b Rename subject_name in send_message_backend(). 2017-10-27 10:48:11 -07:00
rht 45e8ce559d zerver/views: Use python 3 syntax for typing. 2017-10-26 21:58:22 -07:00
Tim Abbott 8e2cdedf9a lint: Fix lines in Python codebase longer than 120 characters. 2017-10-26 17:47:30 -07:00
Tim Abbott b936e8c24b lint: Fix lines in Python codebase longer than 125 characters. 2017-10-26 17:36:54 -07:00
Steve Howell df93a99b50 Cache only one row per message.
Before this change, we populated two cache entries for each
message that we sent.  The entries were largely redundant,
with the only difference being whether we sent the content
as raw markdown or as the rendered HTML.

This commit makes it so we only have one cache entry per
message, and it includes both content and rendered_content.

One legacy source on confusion here is that `content`
changes meaning when you're on the front end.  Here is the
situation going forward:

    database:
        content = raw
        rendered_contented = rendered

    cache entry:
        content = raw
        rendered_contented = rendered

    payload for the frontend:
        content = raw (for apply_markdown=False)
        content = rendered (for apply_markdown=True)
2017-10-26 16:35:28 -07:00
derAnfaenger 1792dcbd09 tests: Call real consume method of queue processors.
This switches to more real tests for a first batch of
queue_json_publish() calls that don't cause trouble when
used with call_consume_tests=True.
2017-10-26 14:58:03 -07:00
Tim Abbott b8658c6901 default stream groups: Use cleaner system for error handling.
Wherever possible, we always want to move checking for error
conditions to the views code, so that we don't need to worry about
handling failures with (in this case) a user that's half-created
because a DefaultStreamGroup doesn't exist.
2017-10-26 11:38:27 -07:00
Vishnu Ks eecdc5bb61 registration: Add option to choose default stream groups.
This effectively implements the feature of default stream groups,
except for a UI, nice styling, etc.

Note that we're careful to not have this do anything in an
organization that doesn't have any default stream groups.
2017-10-26 11:34:06 -07:00
Greg Price 093bae4bc5 subdomains: Fix some implicit uses of "" for the root subdomain.
These are just instances that jumped out at me while working on the
subdomains code, mostly while grepping for get_subdomain call sites.
I haven't attempted a comprehensive search, and there are likely
still others left.
2017-10-26 10:29:17 -07:00
Abhijeet Kaur a88178afaf Embedded bots: Add support for creating embedded bots via the API.
Adds support to add "Embedded bot" Service objects. This service
handles every embedded bot.

Extracted from "Embedded bots: Add support to add embedded bots from
UI" by Robert Honig.

Tweaked by tabbott to be disabled by default.
2017-10-25 16:13:29 -07:00
derAnfaenger d0c0b148b3 bot services: Rename add_outgoing_webhook_service to add_service. 2017-10-25 15:46:07 -07:00
darshanime a14ec17473 dev_login: Don't redirect to realm if it doesn't exist.
Fixes #7132.
2017-10-25 15:13:20 -07:00
Vishnu Ks 1d94119d31 actions: Call send_initial_pms from process_new_human_user. 2017-10-25 14:14:59 -07:00
Tim Abbott 4439dbf781 log_into_subdomain: Fix realm creation on servers using an IP address.
While it's totally fine to put a leading '.' before the cookie domain
for normal hostnames and browsers will just strip them, if you're
using an IP address, it doesn't work, because .127.0.0.1 (for example)
is just invalid, and the cookie won't be set.

This fixes an issue where after installing with an IP address, realm
creation would end with being stuck at a blank page for
/accounts/login/subdomain/.
2017-10-24 14:13:31 -07:00
Tim Abbott 3ee53d5de3 auth: Don't offer password reset links when useless.
If an organization doesn't have the EmailAuthBackend (which allows
password auth) enabled, then our password reset form doesn't do
anything, so we should hide it in the UI.
2017-10-24 12:07:43 -07:00
Tim Abbott b81bde3a90 ldap: Prevent creating accounts with Zulip/EmailAuthBackend passwords.
While our recent changing to hide /register means we don't need a nice
pretty error message here, eventually we'll want to clean up the error
message.

Fixes #7047.
2017-10-24 11:21:49 -07:00
Tim Abbott 914e6a38fe registration: Extract login_and_go_to_home.
This new function extractions the bit of logic we use after creating a
new user account to log them in and send them to the home page,
without emailing the user about their new login.
2017-10-24 11:21:49 -07:00
Tim Abbott 70d509196d backends: Call send_initial_pms on other user creation paths.
This fixes a problem we've seen where LDAP users were not getting this
part of the onboarding process, and a similar problem for human users
created via the API.

Ideally, we would have put these fixes in process_new_human_user, but
that would cause import loop problems.
2017-10-24 09:03:53 -07:00
Steve Howell a1d887c950 Use check_bool validator in get_messages_backend().
This is the more modern way in our codebase to validate
bool fields.
2017-10-20 15:51:57 -07:00
Steve Howell 0e106a2488 Add client_gratavar support to GET /messages.
Clients fetching messages can now specify that they are able
to compute their avatar, and if they set client_gratavar to
True in the request (w/our normal encoding scheme), then the
backend will not compute it, and the payload will be smaller.

The fix starts with get_messages_backend.  The flag gets
passed down through these functions:

    * MessageDict.post_process_dicts.
    * MessageDict.set_sender_avatar.

We also fix up the callers for post_process_dicts to explicitly
pass in the client_gravatar path, but for now they all just hard
code the value to False.
2017-10-20 15:49:21 -07:00
Umair Khan 2172185d5a i18n: Store language name in session; not locale.
In the UI we use locale as the code for the language. Django expects
language code. For Simplified Chinese, 'zh_Hans' is the locale which
maps to a directaory under static/locale, and 'zh-hans' is the language
code, which is used in settings.LANGUAGES setting found in Django.
2017-10-19 17:56:49 -07:00
Tim Abbott 642086ff65 subdomains: Pass whether root domain is available to registration.
This data is necessary to determine whether to offer the root domain
in the realm creation form.
2017-10-18 22:53:10 -07:00
Tim Abbott 1ab2ca5986 subdomains: Extract zerver.lib.subdomains library.
These never really belonged with the rest of zerver.lib.utils.py, and
having a separate library makes it easier to enforce full test
coverage.
2017-10-18 22:27:48 -07:00
Tim Abbott f998550fb4 reactions: Add missing mypy import for Message. 2017-10-18 08:59:29 -07:00
Harshit Bansal 31ce96ff2a views/reactions.py: Extract `create_historical_message()` function. 2017-10-17 22:24:34 -07:00
Alena Volkova 5515a075ec urls: Move the report endpoints to be API-style routes. 2017-10-17 22:05:56 -07:00
Steve Howell d909355dc2 refactor: Move methods from models.py -> lib/messages.py.
Message.get_raw_db_rows is moved to MessageDict, since its
implementation details are highly coupled to other methods
in MessageDict.

And then sew_messages_and_reactions comes along for the
ride.

We eventually want to move Reaction.get_raw_db_rows to there
as well.
2017-10-16 23:37:10 -07:00
Tim Abbott a466bf8a47 report: Fix handling of reports with no more_info. 2017-10-12 17:38:28 -07:00
Steve Howell a6c3621f55 Support client_gravatar in /json/users endpoint. 2017-10-12 14:00:41 -07:00
derAnfaenger 7deb698e02 email_log.py: Remove redundant shebang. 2017-10-12 11:18:45 -07:00
Tim Abbott 339e206c90 highlight_html_differences: Improve logging output.
Now at least it will give the message ID, and thus be possible to
debug.
2017-10-11 23:38:29 -07:00
Tim Abbott 065033e864 auth: Fix mypy annotations in authenticate_remote_user. 2017-10-11 23:32:11 -07:00
Tim Abbott 57b8bfe84b authenticate_remote_user: Accept a realm object.
This paves the path for further code cleanup migrations.
2017-10-11 23:27:00 -07:00
Tim Abbott db645f5825 finish_google_oauth2: Depend on a realm existing. 2017-10-11 23:27:00 -07:00
Tim Abbott e48abcfcf1 auth: Fix confusing realm variable name in JWT auth. 2017-10-11 23:27:00 -07:00
Tim Abbott 6176d0fbca json: Replace most use of simplejson with json.
This is progress towards removing simplejson as a dependency.
2017-10-11 22:55:35 -07:00
Tim Abbott b3b5d5b7cd report: Avoid sending raw message content in error reporting.
This fixes a violation of Zulip's privacy policies (that error
reporting never contain message content) in the previous commit.
2017-10-11 17:44:05 -07:00
Steve Howell 3910448b1d Extract MessageDict.post_process_dicts().
Introduce MessageDict.post_process_dicts() will allow us
the ability to do the following:

    * use less memory in the cache for repeated data
    * prevent cache invalidation
    * format data according to different client needs

The first use of this function is pretty inconsequential, but
it sets us up for more consequential changes.

In this commit we defer the MessageDict.hydrate_recipient_info
step until after we pull data out of the cache.  This impacts
cache size as follows:

    * streams - negligibly bigger
    * PMs/huddles - slimmer due to not needing to repeat
                    sender data like email/full_name

Again, the main point of this change is to start setting up
the infrastructure to do post-processing.
2017-10-11 11:37:16 -07:00
Vishnu Ks 63ed4110fe auth: Submit dev login form to the correct subdomain. 2017-10-11 10:16:55 -07:00
Alena Volkova 79560e21bf urls: Move the json/tutorial_status endpoint to be an API-style route. 2017-10-09 15:13:33 -07:00
Steve Howell 75db4f5187 Avoid O(N) query pitfall for /users endpoint.
We now use a `.values` query to get just the fields we need
in order to fulfill '/json/users' requests.

The main benefit is that we don't do O(N) queries for bot
owners, but we also have less data on UserProfile to process.
2017-10-09 14:08:07 -07:00
Harshit Bansal 3acaa79336 reactions.py: Don't check for valid emoji name while removing reaction.
On receiving a request for deleting a reaction, just check if such
a reaction exists or not. If it exists then just delete the reaction
otherwise send an error message that such a reaction doesn't exist.
It doesn't make sense to check whether an emoji name is valid or not.
2017-10-09 17:54:37 +00:00
Steve Howell a67b07bfc4 Avoid shadowing stream variables.
This helps with mypy typings in an upcoming commit.
2017-10-08 20:18:34 -07:00
Steve Howell 7dbea8a2bf Only require stream_id in subscribed_to_stream().
Since subscribed_to_stream is only doing an id lookup
on the Stream model to find out if a user is subscribed to
a stream, there's no reason to require a full Stream object.

It's currently the case that all callers do have full Stream
objects handy to pass in to this function, but it's still a
good practice to have functions only ask for objects that they
need.
2017-10-08 20:18:34 -07:00
Greg Price a116303604 passwords: Express the quality threshold as guesses required.
The original "quality score" was invented purely for populating
our password-strength progress bar, and isn't expressed in terms
that are particularly meaningful.  For configuration and the core
accept/reject logic, it's better to use units that are readily
understood.  Switch to those.

I considered using "bits of entropy", defined loosely as the log
of this number, but both the zxcvbn paper and the linked CACM
article (which I recommend!) are written in terms of the number
of guesses.  And reading (most of) those two papers made me
less happy about referring to "entropy" in our terminology.
I already knew that notion was a little fuzzy if looked at
too closely, and I gained a better appreciation of how it's
contributed to confusion in discussing password policies and
to adoption of perverse policies that favor "Password1!" over
"derived unusual ravioli raft".  So, "guesses" it is.

And although the log is handy for some analysis purposes
(certainly for a graph like those in the zxcvbn paper), it adds
a layer of abstraction, and I think makes it harder to think
clearly about attacks, especially in the online setting.  So
just use the actual number, and if someone wants to set a
gigantic value, they will have the pleasure of seeing just
how many digits are involved.

(Thanks to @YJDave for a prototype that the code changes in this
 commit are based on.)
2017-10-08 15:48:44 -07:00
Tim Abbott 43bba27e5d auth: Fix redirect loop in development environment.
Since the REALMS_HAVE_SUBDOMAINS migration in development, we've had
scattered reports of users who found trying to open 127.0.0.1:9991
resulting in a redirect loop between zulipdev.com:9991,
zulipdev.com:9991/devlogin, and zulipdev.com:9991/devlogin/, and back
to zulipdev.com:9991.

We fix this temporarily through a small cleanup, which is to have that
last step in the loop send the user to the subdomain where they're
actually logged in, zulip.zulipdev.com:9991.

There's more to be done before this system will make sense, though.
2017-10-05 23:21:29 -07:00
Alena Volkova 21bc0e845c urls: Delete the json/tutorial_send_message endpoint.
This endpoint is part of the old tutorial, which we've removed, and
has some security downsides as well.

This includes a minor refactoring of the tests.
2017-10-05 15:41:57 -07:00
Tim Abbott e1eb91343b zephyr: Update path to process_cache for repository split.
This should have been done long ago, but better late than never.
2017-10-05 12:35:20 -07:00
Tim Abbott 636c9429f6 email_log: Don't require full coverage for error handling path. 2017-10-05 10:34:56 -07:00
Steve Howell c567f105c9 Have topic_is_muted take a stream_id.
This function doesn't need a full Stream object to detect
whether a stream is muted, so we can save future callers
from doing unnecessary DB fetches.
2017-10-05 09:32:16 -07:00
Vishnu Ks cad9ad85d7 email_log: Make sure email generation requests are successful. 2017-10-04 14:45:08 -07:00
Tim Abbott ce7ab0474d error_notify: Add IP address to browser error reports.
This should make debugging a bit more convenient when we want to know
which of a user's clients was involved.
2017-10-04 13:46:05 -07:00
Vishnu Ks 80fc9cda3a settings: Mention about /emails after sending change email mail. 2017-10-04 08:20:29 -07:00
Vishnu Ks 4c2e787ffc emails: Redirect to /emails after generating emails. 2017-10-04 08:20:29 -07:00
Steve Howell aae0b2a826 Notify offline users about edited stream messages.
We now do push notifications and missed message emails
for offline users who are subscribed to the stream for
a message that has been edited, but we short circuit
the offline-notification logic for any user who presumably
would have already received a notification on the original
message.

This effectively boils down to sending notifications to newly
mentioned users.  The motivating use case here is that you
forget to mention somebody in a message, and then you edit
the message to mention the person.  If they are offline, they
will now get pushed notifications and missed message emails,
with some minor caveats.

We try to mostly use the same techniques here as the
send-message code path, and we share common code with the
send-message path once we get to the Tornado layer and call
maybe_enqueue_notifications.

The major places where we differ are in a function called
maybe_enqueue_notifications_for_message_update, and the top
of that function short circuits a bunch of cases where we
can mostly assume that the original message had an offline
notification.

We can expect a couple changes in the future:

    * Requirements may change here, and it might make sense
      to send offline notifications on the update side even
      in circumstances where the original message had a
      notification.

    * We may track more notifications in a DB model, which
      may simplify our short-circuit logic.

In the view/action layer, we already had two separate codepaths
for send-message and update-message, but this mostly echoes
what the send-message path does in terms of collecting data
about recipients.
2017-10-03 15:57:06 -07:00
Cynthia Lin cfeb125624 zerver: Block access to subdirectories when visiting user docs articles.
Fixes #6770
2017-10-03 11:16:09 -07:00
Tim Abbott 1d72629dc4 subdomains: Hardcode REALMS_HAVE_SUBDOMAINS=True. 2017-10-02 16:42:43 -07:00
Tim Abbott 13bb546ddd subdomains: Hardcode can_add_realm_domains=True.
It was already always True if REALMS_HAVE_SUBDOMAINS, and now that's
the only supported configuration.
2017-10-02 16:32:10 -07:00
Tim Abbott 66658bbf25 subdomains: Hardcode REALMS_HAVE_SUBDOMAINS in API docs. 2017-10-02 16:32:10 -07:00
Tim Abbott 262a0692ab subdomains: Delete completely_open helper function.
We no longer use this anyway.
2017-10-02 16:32:10 -07:00
Tim Abbott e6f8032972 subdomains: Remove get_unique_open_realm code paths.
Since we no longer support !REALMS_HAVE_SUBDOMAINS in production,
these no longer make sense.
2017-10-02 16:32:10 -07:00
Vishnu Ks 6b9755c854 email_log: Make generate_email work with subdomains.
This fixes one of our last few test failures with
REALMS_HAVE_SUBDOMAINS.
2017-10-02 12:27:27 -07:00
Umair Khan 69ccc8ce0e ldap: Show helpful message when realm is None. 2017-09-30 10:18:25 -07:00
Tim Abbott 0744961afe docs: Add more comprehensive documentation on error reporting.
Fixes #2232.
2017-09-27 19:47:08 -07:00
rht e4c824ed61 zerver/views: Remove print_function. 2017-09-27 18:05:45 -07:00
Vishnu Ks 94e099eaab auth: Make multiuse invite link work with oAuth2.
This works by attaching to the user's session the multi-use invitation
key, allowing that to be used in the Google/GitHub auth flows.
2017-09-27 17:47:40 -07:00
rht 106375bd68 zerver/views: remove `import six`. 2017-09-27 17:03:03 -07:00
Steve Howell 646abb57b7 refactor: Extract get_user_info_for_message_updates.
We'll want to expand this to get users that were mentioned in
the prior message, but this commit is just a refactoring.
2017-09-27 16:01:50 -07:00
rht 2949d1c1e8 zerver: Remove the rest of absolute_import. 2017-09-27 10:02:39 -07:00
rht 15ca13c8de zerver/views: Remove absolute_import. 2017-09-27 10:00:39 -07:00
Tim Abbott 1c25bb5eba auth: Fix typo in invalid subdomain logging. 2017-09-26 22:20:05 -07:00
Vishnu Ks 7fd4a71b7f test_emails: Generate emails instead of hardcoding.
Tweaked by tabbott to use require_GET.

Fixes #6344.
2017-09-26 15:02:47 -07:00
Alena Volkova ca687e01d7 urls: Remove the old POST endpoint for alert words. 2017-09-26 14:00:51 -07:00
neiljp (Neil Pilgrim) ef1242bb62 mypy: Split email validation error handling in json_change_settings. 2017-09-25 16:02:56 -07:00
Tim Abbott 0e1518bb67 auth: Disable coverage check for an auth error path.
This is just a temporary change to get tests passing again, though
it's possible we'll be deleting this code path before long.
2017-09-25 12:06:10 -07:00
Vishnu Ks b4fedaa765 backend: Add support for multiuse user invite link. 2017-09-22 07:56:53 -07:00
Vishnu Ks 68ccfe78e6 registration: Add optional streams argument to send_registration_completion_email. 2017-09-22 07:51:58 -07:00
Tim Abbott 1edd137263 RealmAuditLog: Pass acting_user to do_reactivate_user. 2017-09-22 07:33:02 -07:00
Rishi Gupta 88bec16452 hotspots: Only narrow to PMs on first load.
After first load, the browser should set tutorial-status to started,
so this should automatically only happen once.

Fixes #6575.
2017-09-22 05:01:53 -07:00
Steve Howell f13cb94d84 Have exclude_topic_mutes() accept a stream id.
We want to convert stream names to stream ids as close
to the "edges" of our system as possible, so we let our
caller do the work of finding the stream id for a stream
narrow.
2017-09-20 10:31:33 -07:00
Steve Howell 26735eeeac Only require realm_id for get_active_user_dicts_in_realm().
This is a preparatory commit that will eventually allow us
to avoid fetching realm info that we don't need, in other
parts of the codebase.
2017-09-20 10:31:33 -07:00
Tim Abbott 15781dbc83 server_settings: Add email auth related features to data sent to clients.
This should make it possible for the mobile app to correctly allow
non-email addresses as usernames exactly when it makes sense to do so.
2017-09-15 10:30:20 -07:00
Sarah c3a8138f74 user_settings: Add push notifications for all stream messages.
Add setting to enable push notifications for all stream messages.
2017-09-14 05:41:37 -07:00
Steve Howell ba397b5109 Use user_ids, not full objects, in render path.
There is no reason for either render_incoming_message() or
render_markdown() to require full UserProfile objects just to
triage alert words.

By only asking for user_ids, we save extra queries in two
callpaths and we make it easier to start using user_ids in
do_send_messages().
2017-09-12 04:22:55 -07:00
Steve Howell 4ac6bc46c7 Add MutedTopic model.
This commit completely switches us over to using a
dedicated model called MutedTopic to track which topics
a user has muted.

This includes the necessary migrations to create the
table and populate it from legacy data in UserProfile.

A subsequent commit will actually remove the old field
in UserProfile.
2017-09-02 09:19:51 -07:00
Steve Howell 0501570cd1 Remove POST-based API for setting topic mutes. 2017-08-29 16:53:38 -04:00
Steve Howell 8c4a5a9f7a Extract exclude_topic_mutes.
This is mostly a pure code move, but I cleaned up the code
slightly to use early-return.
2017-08-29 16:53:38 -04:00
Tim Abbott f1ad819547 home: Remove compatibility code for old name Humbug.
Since Zulip stopped being called Humbug in like 2013, this code hasn't
been useful in years, and is a bit confusing.
2017-08-28 16:15:58 -07:00
Tim Abbott ac0d90e533 portico: Rename 'find_my_team' to 'find_account'. 2017-08-28 14:29:29 -07:00
Tim Abbott a0a1fe1512 settings: Rename SERVER_URI to ROOT_DOMAIN_URI.
This should be a lot less confusing.

See #6013 for discussion.
2017-08-28 14:09:28 -07:00
Umair Khan ecfafc05c0 registration: Use already_registered to show error.
Use this new variable to determine if the user already exists while
doing registration. While doing login through GitHub if we press
*Go back to login*, we pass email using email variable. As a result,
the login page starts showing the "User already exists error" if we
don't change the variable.
2017-08-28 07:02:11 -07:00
Steve Howell 313f73258d Allow admins to delete private streams (backend only).
This is the backend piece.  Getting the UI right here is a bit
more complicated here, but this allows admins to use the API
to delete streams.
2017-08-27 19:08:04 -07:00
Umair Khan 9720d3a986 accounts_register: Use redirect_to_deactivation_notice. 2017-08-25 00:09:06 -07:00
Umair Khan b473ff54ce Show realm deactivation notice for login/register.
Fixes #6247
2017-08-25 00:09:06 -07:00
Tim Abbott e2ae096035 test_signup: Fix coverage of remaining find_my_team code. 2017-08-24 23:30:46 -07:00
Tim Abbott 59aae22f99 registration: Fix find_my_team handling of unusual users. 2017-08-24 23:17:08 -07:00
Tim Abbott ffafa05bd5 decorator: Remove unnecesary import. 2017-08-24 23:16:31 -07:00
Tim Abbott b3dbba3ad4 actions: Extract validate_email_for_realm helper. 2017-08-24 23:16:31 -07:00
Tim Abbott 60105fe38a registration: Remove accounts_home_with_realm_str.
This was the old mechanism for logging into a specific realm on a
server with more than one not using REALMS_HAVE_SUBDOMAINS.
2017-08-24 20:44:52 -07:00
Tim Abbott 3d4893d906 settings: Rename SUBDOMAINS_HOMEPAGE to ROOT_DOMAIN_LANDING_PAGE.
This new setting name is a lot more readable.
2017-08-24 19:32:16 -07:00
Tim Abbott ab61fe5922 auth: Fix Google auth to not assume SUBDOMAINS_HOMEPAGE.
Previously, Google auth would redirect you back to the login page on
this code path.
2017-08-24 19:03:42 -07:00
Tim Abbott 8b0c0c33b5 login: Fix logging for login/register actions.
Previously, Zulip's server logs would not show which user or client
was involved in login or user registration actions, which made
debugging more annoying than it needed to be.
2017-08-24 16:29:27 -07:00
Tim Abbott a570a57fd5 login: Log which user was authenticated in mobile flows. 2017-08-24 16:29:27 -07:00
Steve Howell 87c4961597 Add zerver/lib/topic_mutes.py
This is mostly pure code extraction.

It also removes some dead code in update_muted_topic, where
were updating muted_topics spuriously before calling
do_update_muted_topic.
2017-08-24 14:20:35 -07:00
Tim Abbott f46f3871b4 integrations: Add support for using root subdomain.
This fixes up the URL/link construction to make the root subdomain work.
2017-08-23 23:18:36 -07:00
Tim Abbott d3e3c704d4 do_create_realm: Remove unnecessary second return value.
Unlike creating a stream, there's really no reason one would want to
call the function to create a realm while uncertain whether that realm
already existed.
2017-08-23 20:07:17 -07:00
Tim Abbott 8a65fdce49 subdomains: Fix over-aggressive redirect to find_my_team.
If the root domain is a valid subdomain, we don't want to redirect
everyone away from it.
2017-08-23 20:07:17 -07:00
Vishnu Ks 8418f0d30e dev_login: List realms and show only users in the selected realm. 2017-08-23 19:27:32 -07:00
Vishnu Ks 50fe1f7d9c streams: Fix errors when notifications_stream is deactivated.
Fixes #6225.
2017-08-23 17:54:05 -07:00
Steve Howell 2b4faaa847 Support non-search queries in /json/messages/matches_narrow.
For filters like has:link, where the web app doesn't necessarily
want to guess whether incoming messages meet the criteria of the
filter, the server is asked to query rows that match the query.

Usually these queries are search queries, which have fields for
content_matches and subject_matches.  Our logic was handling those
correctly.

Non-search queries were throwing an exception related to tuple
unpacking.  Now we recognize when those fields are absent and
do the proper thing.

There are probably situations where the web app should stop hitting
this endpoint and just use its own filters.  We are making the most
defensive fix first.

Fixes #6118
2017-08-23 01:07:57 -07:00
Tim Abbott 06f8fd529d realm: Add clear error message for too-long realm name.
This fixes an exception we saw in production, as well as fixing the
form to not offer ultra-long names that we don't allow.
2017-08-22 22:04:52 -07:00
Tim Abbott 7c37fc8909 auth: Improve robustness of not sending duplicate login emails.
Previously, we had a very not-robust check on the URL, which might
have caused cases like Google auth registration to not do the right
thing.
2017-08-22 16:26:30 -07:00
Rishi Gupta d296f29fa7 emails: Link to organization getting started guide in followup_day1. 2017-08-22 14:26:17 -07:00
Rishi Gupta 7c3f20d2ba registration: Set is_realm_admin on user creation.
This makes it easier for later parts of the user creation/onboarding process
to condition on whether the user is a realm admin.

No change in behavior.
2017-08-22 14:26:17 -07:00
Steve Howell 54edecd510 Replace adddressee.for_email() with for_user_profile().
This requires us to change not just the immediate caller, but
also some of their callers, to pass user_profile objects around
instead of emails.
2017-08-22 10:42:15 -07:00
Steve Howell e12da79732 create_realm: Redirect to a nice error page for SMTP errors.
Fixes #3850
2017-08-21 15:39:26 -07:00
Steve Howell 99e08fe49a accounts_home: Redirect to nice error page for SMTP errors. 2017-08-21 15:39:26 -07:00
Greg Price a099e698e2 py3: Switch almost all shebang lines to use `python3`.
This causes `upgrade-zulip-from-git`, as well as a no-option run of
`tools/build-release-tarball`, to produce a Zulip install running
Python 3, rather than Python 2.  In particular this means that the
virtualenv we create, in which all application code runs, is Python 3.

One shebang line, on `zulip-ec2-configure-interfaces`, explicitly
keeps Python 2, and at least one external ops script, `wal-e`, also
still runs on Python 2.  See discussion on the respective previous
commits that made those explicit.  There may also be some other
third-party scripts we use, outside of this source tree and running
outside our virtualenv, that still run on Python 2.
2017-08-16 17:54:43 -07:00
Tim Abbott 7cfb4e195f do_deactivate_user: Add acting user to RealmAuditLog. 2017-08-16 16:23:41 -07:00
Steve Howell 47bcedbc1b Fix server searches for is:mentioned and is:alerted.
Before this change, server searches for both
`is:mentioned` and `is:alerted` would return all messages
where the user is specifically mentioned (but not
at-all mentions).

Now we follow the JS semantics:

    is:mentioned -- all mentions, including wildcards
    is:alerted  -- has an alert word

Here is one relevant JS snippet:

        } else if (operand === 'mentioned') {
            return message.mentioned;
        } else if (operand === 'alerted') {
            return message.alerted;

And here you see that `mentioned` is OR'ed over both mention flags:

    message.mentioned = convert_flag('mentioned') || convert_flag('wildcard_mentioned');

The `alerted` flag on the JS side is a simple mapping:

    message.alerted = convert_flag('has_alert_word');

Fixes #5020
2017-08-16 11:29:12 -07:00
Vaida Plankyte 52046d537a auth.py: Add config_error page for misconfigured github/google auth.
Significantly modified by tabbott to use a better system, pass tests,
and clean up the content.
2017-08-16 10:05:19 -07:00
neiljp (Neil Pilgrim) 2d3b95a088 mypy: Annotate sent_time_in_epoch_seconds taking Optional[UserMessage]. 2017-08-15 17:50:18 -07:00
Brock Whittaker 83184d5c60 /about/: Merge with /authors, redesign page.
This adds the authors to the Zulip repository on GitHub from
/authors/ along with re-styling the page to fit the same
aesthetic as /for/open-source/ and other product-pages.
2017-08-15 16:09:01 -07:00
Jack Zhang b82bdc82bf context_processors.py: Add flag for whether page is help center.
This is needed once the header markup is deduplicated.
2017-08-15 12:15:21 -07:00
Tim Abbott ffc1ceeaa4 lint: Ban use of get_stream in the rest of messages.py.
This will help avoid potential future security bugs.
2017-08-15 10:40:02 -07:00
Tim Abbott 842bf77efb unread: Convert mark_topic_as_read to use stream IDs.
This is safer in its handling of potentially renamed streams.
2017-08-15 10:40:02 -07:00
Tim Abbott f3a41ac796 unread: Convert mark_stream_as_read to use stream IDs.
The logic is simpler and more robust.
2017-08-15 10:40:02 -07:00
Steve Howell 89f9017686 Create new endpoints for marking streams/topics as read.
The new endpoints are:
    /json/mark_stream_as_read: takes stream name
    /json/mark_topic_as_read: takes stream name, topic name

The /json/flags endpoint no longer allows streams or topics
to be passed in as parameters.
2017-08-15 10:17:29 -07:00
Steve Howell 60cc8fd58a Extract do_mark_stream_messages_as_read.
This function optimizes marking streams and topics as read,
by using UserMessage.where_unread(), which uses a partial
index on the "read" flag.

This also simplifies the code path for ordinary message
flag updates.

In order to keep 100% line coverage, I simplified the
logging in update_message_flags, so now all requests
will show the "actually" format.

This is an interim step toward creating dedicated endpoints
for marking streams/topics as reads, so we do error checking
with asserts for flag/operation, so we don't introduce a
temporary translation string.
2017-08-15 10:09:10 -07:00
Steve Howell ffe4f09c88 Extract update_stream_topic_message_flags.
This is mostly a pure code extraction, except that we now
disregard the `messages` option for stream/topic updates,
since the web app always passes in an empty list (and this
commit is really just an incremental step toward creating
new endpoints.)
2017-08-15 10:09:10 -07:00
Tim Abbott a6a3915dd7 messages: Move is_public_stream out of views code.
The main purpose of this change is to minimize access to get_stream in
views code.
2017-08-15 10:06:26 -07:00
Steve Howell 274938bf5f Eliminate approximate_unread_count().
We use page_params.unread_msgs.count in the web
app instead.

Fixes #1300.
2017-08-14 12:39:00 -07:00
Jack Zhang fc6c2b321b portico: Reimplement /apps routes using pushState. 2017-08-10 10:21:53 -07:00
Jack Zhang 1e6570fa5d Update integrations doc context relative urls. 2017-08-10 10:21:53 -07:00
Tim Abbott 1d10941712 register: Don't display field to enter password unless needed.
This should significantly improve the user experience for new users
signing up with GitHub/Google auth.  It comes complete with tests for
the various cases.  Further work may be needed for LDAP to not prompt
for a password, however.

Fixes #886.
2017-08-09 13:44:57 -07:00
Umair Khan 4a3fde023f google: Respect is_signup argument.
This allows us to go to Registration form directly. This behaviour is
similar to what we follow in GitHub oAuth. Before this, in registration
flow if an account was not found, user was asked if they wanted to go to
registration flow. This confirmation behavior is followed for login
oauth path.
2017-08-09 13:44:57 -07:00
Umair Khan bb0eb76bf3 github: Don't ask for password in registration. 2017-08-09 13:44:57 -07:00