Commit Graph

2515 Commits

Author SHA1 Message Date
Aditya Bansal f46d098558 schedulemessages: Add handle_deferred_message() to handle requests.
This is responsible for:
1.) Handling all the incoming requests at the
messages endpoint which have defer param set. This is similar to
send_message_backend apart from the fact that instead of really
sending a message it schedules one to be sent later on.
2.) Does some preliminary checks such as validating timestamp for
scheduling a message, prevent scheduling a message in past, ensure
correct format of message to be scheduled.
3.) Extracts time of scheduled delivery from message.
4.) Add tests for the newly introduced function.
5.) timezone: Add get_timezone() to obtain tz object from string.
This helps in obtaining a timezone (tz) object from a timezone
specified as a string. This string needs to be a pytz lib defined
timezone string which we use to specify local timezones of the
users.
2018-01-19 11:33:11 -05:00
rht ee06da9350 bugdown: Linkify Bitcoin URI. 2018-01-19 06:07:43 -05:00
Shubham Dhama 797fa62fde test auth.py: Add tests for json_fetch_api_key function. 2018-01-17 15:15:51 -05:00
Rhea Parekh 138aba3d89 slack importer: Add unit tests for slack message conversions. 2018-01-17 09:38:53 -05:00
Shreyansh Dwivedi b0fb7aa6b2 bugdown: Add preview for vimeo videos.
This also amends a commit from Brock Whittaker <brock@zulipchat.com>
that merges two separate functions for YouTube videos and Vimeo videos
into a generic video recall function.

Fixes #7550.
2018-01-17 09:11:43 -05:00
Xavier Cooney 8c5e74afe9 test_messages.py: Remove unecessary imports. 2018-01-16 08:16:43 -05:00
Eeshan Garg a14b52ef1d api docs: Document the `DELETE /users/me/subscriptions` endpoint.
Note that there is currently no JavaScript method in zulip-js
to call this endpoint.
2018-01-12 07:30:38 -05:00
Eeshan Garg 8c723e8c27 api docs: Document the POST /api/v1/users endpoint.
Note that there is no JavaScript equivalent for requesting this
endpoint in zulip-js yet.
2018-01-12 07:30:38 -05:00
Andy Perez c209069fcd bugdown: Improve image inlining logic.
Fix #7537
2018-01-09 16:43:22 -05:00
Robert Hönig d1d8365a6b embedded bots: Add config data UI.
This adds UI fields in the bot settings for specifying
configuration values like API keys for a bot. The names
and placeholder values for each bot's config fields are
fetched from the bot's <bot>.conf template file in the
zulip_bots package. This also adds giphy and followup
as embedded bots.
2018-01-09 11:50:54 -05:00
Alena Volkova 7c22045158 test_decorators: Replace the json/fetch_api_key endpoint in tests.
This endpoint is about to become an API-style route and have the legacy
decorator removed from its view. Other endpoints will be used in tests
instead of it.
2018-01-08 13:15:52 -05:00
Robert Hönig 3420ed1fcc validator.py: Cover missing lines in check_dict(). 2018-01-07 15:23:17 -05:00
Eeshan Garg a01f8c088a api docs: Document the POST /api/v1/users/me/subscriptions endpoint.
There is a JavaScript equivalent for this endpoint but the npm
package has not yet been released.
2018-01-07 15:19:51 -05:00
Eeshan Garg ae59b08316 api docs: Document the GET /api/v1/users/me endpoint.
Note that there is no JavaScript equivalent for requesting this
endpoint in zulip-js yet.
2018-01-07 15:19:51 -05:00
Eeshan Garg 045aacbc67 api docs: Document the PATCH /api/v1/messages/<msg_id> endpoint.
Note that there is no JavaScript equivalent for requesting this
endpoint in zulip-js yet.
2018-01-07 15:19:51 -05:00
Eeshan Garg 6f01b042a1 api docs: Document the DELETE /api/v1/events endpoint.
Note that there is no JavaScript equivalent for requesting this
endpoint in zulip-js yet.
2018-01-07 15:19:51 -05:00
Robert Hönig c745d1901c bot_config.py: Add function for loading config file templates.
load_bot_config_template(bot) parses the <bot>.conf
template file, which can be found in the zulip_bots
package for each bot. It then returns the INI content
of that file as a dict.
2018-01-07 20:05:52 +01:00
Robert Hönig ed7208fd6e Make EmbeddedBotHandler.get_config_info comply with ExternalBotHandler. 2018-01-07 20:05:52 +01:00
Robert Hönig 9cc059f4aa bot_config.py: Raise exception when getting config data from bot without config entries. 2018-01-07 20:05:52 +01:00
Eeshan Garg 479bc41ca5 api docs: Remove old and outdated endpoint docs. 2018-01-04 10:17:29 -05:00
Eeshan Garg 1fd71287d6 api docs: Document the GET /api/v1/events endpoint. 2018-01-04 10:17:29 -05:00
Vishnu Ks e6d3f8895f messages: Create function to calculate first visible message id. 2018-01-04 08:52:40 -05:00
Eeshan Garg f620525373 api docs: Document the POST /api/v1/register endpoint. 2018-01-03 12:23:30 -05:00
Alena Volkova 45f0c76c44 settings: Limit the creation of generic bots.
This commit adds a setting to limit creation of generic bots
to admins for realms that want that restriction.  (Generic
bots, apart from being considered spammy on some realms,
have less locked down permissions than webhook bots).

Fixes #7066.
2018-01-02 18:12:22 -05:00
Aastha Gupta daf86eb664 settings: Add "text" option to emoji_set model.
We no longer have a special UI setting and model
field ("emoji_alt_code") for saying users want text-only
emojis.  We now instead make "text" be a fifth choice
for "emojiset".

Fixes #7406
2018-01-02 14:55:01 -05:00
Vishnu Ks 277d463d23 messages: Add option to limit users message history. 2018-01-02 13:45:01 -05:00
YJDave fa44d2ea69 settings: Remove autoscroll_forever setting.
Fixes #6845
2018-01-02 10:35:49 -05:00
Eeshan Garg 80405747ce test_docs: Test new API docs (templates/zerver/api/*). 2017-12-30 11:03:23 -05:00
Eeshan Garg 94bc9d52d1 api docs: Split usage.md into two separate docs.
This commit splits usage.md into two separate docs,
stream-message.md and private-message.md. The arguments and return
values for sending a stream message are somewhat different from
those of sending a private message, so it made sense to split the
two up for clarity.
2017-12-29 17:03:36 -05:00
Rhea Parekh 3d4ee3b862 zerver/tests/test_email_mirror: Sweep force_str. 2017-12-26 09:09:31 -05:00
Rhea Parekh 092472f42d tests: Add test coverage for uploading avatars to S3UploadBackend.
Add test coverage for the function upload_avatar_image and
ensure_medium_avatar_image in S3UploadBackend.
2017-12-26 09:09:31 -05:00
Viraat Chandra 553711e671 mypy: Use Python 3 syntax for typing in `zerver/tests/test_home.py`. 2017-12-26 08:31:43 -05:00
Xavier Cooney 06b462ba19 mypy: Use Python 3 type syntax in tests/test_auth_backends.py. 2017-12-26 08:29:30 -05:00
Greg Price ac6f3373b6 errors tests: Deduplicate and isolate the magical part.
This makes the tests look a lot more boring and straightforward, in
addition to being a bit shorter.
2017-12-22 11:52:13 -05:00
Greg Price de5c944980 errors tests: De-indent a bunch of code.
This diff is nothing but dedentation -- it's empty under
`git diff -b`.  These with-statements are only needed for
a pretty narrow scope of code, so make that clear in the
source.
2017-12-22 11:52:13 -05:00
Greg Price ca5c991994 errors: Simplify manual testing of error emails.
There are two different things you need to patch in order to get error
emails (at `/emails`) in dev.  Flip one of them in dev all the time,
and make the comment on the other a bit more explicit.
2017-12-22 11:52:13 -05:00
Greg Price 68fa29c3ec errors: Make Zulip and email paths more parallel.
This helps prevent them from diverging and getting different sets of
features and fixes.  As a bonus, the email path gets a nice tweak that
the Zulip path has had for years, since f7f2ec0ac, which makes the
emails clearer and less broken-looking when logging a message with no
stack trace.
2017-12-22 11:52:13 -05:00
Greg Price b15231dfc2 logging: Rename AdminZulipHandler to AdminNotifyHandler.
This name hasn't been right since f7f2ec0ac back in 2013; this handler
sends the log record to a queue, whose consumer will not only maybe
send a Zulip message but definitely send an email.  I found this
pretty confusing when I first worked on this logging code and was
looking for how exception emails got sent; so now that I see exactly
what's actually happening here, fix it.
2017-12-22 11:52:13 -05:00
Tommy Ip 9ea7b05cfe bugdown: Rewrite markdown link to relative link conditionally. 2017-12-22 08:10:42 -05:00
Umair Khan 68513952fb email-worker: Create EmailSendingWorker.
This commit just copies all the code from MissedMessageSendingWorker
class to a new EmailSendingWorker class. All the logic to send an email
through a queue was already there. This commit only makes the logic
generic. It does so by creating a special purpose queue called
'email_senders' to send any type of email. To make
MissedMessageSendingWorker still work we derive it from
EmailSendingWorker. All the tests that were testing
MissedMessageSendingWorker now run against EmailSendingWorker.
2017-12-20 19:36:27 -08:00
Rishi Gupta 869b4d41ef models: Add ScheduledEmail.realm.
The two extra queries in the test are due to the assert in
send_future_email.
2017-12-19 17:46:36 -08:00
Rishi Gupta 87e164043f portico: Rename plans at /plans. 2017-12-19 15:36:13 -08:00
Andy Perez 44e6d9b54b testing: Add nice diffs to markdown python tests. 2017-12-18 19:03:38 -05:00
Greg Price 014900c2e5 push notifs: Include same stream and sender info on GCM as on APNs.
This takes the information added to APNs payloads by #7080, and adds
to our GCM payloads the parts that it didn't already have.
2017-12-18 15:32:56 -08:00
civilnewsnetwork 0ce8b69fd6 Renamed 'Register' to 'Sign up' including translations 2017-12-18 10:02:15 -05:00
Alena Volkova 39915d9835 tests: Replace all get_auth usages with auth methods and delete it. 2017-12-18 09:24:09 -05:00
Alena Volkova 26eb23afc7 tests: Replace api_auth usages with auth methods in zerver/tests. 2017-12-18 09:24:09 -05:00
Umair Khan 07961611b2 test_create_update_and_remove_default_stream_group: Fix flake. 2017-12-14 05:35:54 -05:00
Umair Khan b98c5c6d45 test_multi_user_subscription: Fix flake. 2017-12-14 05:35:54 -05:00
Rhea Parekh dd1b14378b Streams: Disallow certain markdown characters in stream names. 2017-12-12 10:04:31 -06:00
Rhea Parekh 8ed7f73199 IRC mirror: No longer require IRC stream names to start with #. 2017-12-12 10:04:31 -06:00
kunall17 98943a8333 Include data of stream/private sender details in apns push notifications 2017-12-11 20:51:09 -08:00
Shreyansh Dwivedi 47fcb27e39 invitations: Remove custom_body.
Fixes #7672
2017-12-11 19:23:54 -08:00
Rishi Gupta dbe3576706 registration: Enforce realm is None only if realm_creation.
Commit d4ee3023 and its parent have the history behind this code.

Since d4ee3023^, all new PreregistrationUser objects, except those for
realm creation, have a non-None `realm`.  Since d4ee3023, any legacy
PreregistrationUsers, with a `realm` of None despite not being for
realm creation, are treated as expired.  Now, we ignore them
completely, and remove any that exist from the database.

The user-visible effect is to change the error message for
registration (or invitation) links created before d4ee3023^ to be
"link does not exist", rather than "link expired".

This change will at most affect users upgrading straight from 1.7 or
earlier to 1.8 (rather than from 1.7.1), but I think that's not much
of a concern (such installations are probably long-running
installations, without many live registration or invitation links).

[greg: tweaked commit message]
2017-12-11 18:36:14 -08:00
Steve Howell 738c73f08e lint: Prevent `self: Any` annotations.
We should omit these for mypy.  For most class definitions,
mypy doesn't need `Any`, and it provides no real useful info.

For clever monkeypatches, you should provide a more specific
type than `Any`.
2017-12-08 19:00:18 -08:00
Vishnu Ks ff43443ac7 models: Create is_cross_realm_bot_email function. 2017-12-07 14:55:06 -08:00
Rohitt Vashishtha 9c2a28b6df invitations: Fix lint errors in test_queue_workers. 2017-12-06 22:15:25 -08:00
Rishi Gupta 100167fcf8 invitations: Change error messages for resend/revoke views.
We don't have a concept of invitation ID on the frontend (or backend).
2017-12-06 20:35:50 -08:00
Rishi Gupta cb5eb47470 invitations: Move resend/revoke error handling out of actions.py. 2017-12-06 20:35:50 -08:00
Rishi Gupta 968aae167b invitations: Remove get_prereg_user_by_email.
The original logic is buggy now that emails can belong to (and be
invited to) multiple realms.

The new logic in the `invites` queue worker also avoids the bug where
when the PreregistrationUser was gone by the time the queue worker got
to the invite (e.g., because it'd been revoked), we threw an exception.

[greg: fix upgrade-compatibility logic; add test; explain
revoked-invite race above]
2017-12-06 20:35:50 -08:00
Greg Price cab5c418f1 invitations: Add tests for queue worker. 2017-12-06 20:30:42 -08:00
Rishi Gupta 8baadc6b15 invitations: Add check for invite limit to resend invite pathway. 2017-12-06 20:30:42 -08:00
Rishi Gupta e48bc9136a invites: Change resend invitation pathway to use existing code.
This code changes frequently enough that errors are bound to creep in.  The
main change is that this sends the original invitation email instead of the
reminder email, but I think that's fine.
2017-12-06 20:30:42 -08:00
Alena Volkova 3be8e95d13 settings: Send bot types from the backend instead of hardcoding them. 2017-11-30 15:28:53 -08:00
Rishi Gupta 7d1c88f0fb invitations: Limit realms to 100 invites per day.
To guard against using zulip invites as a vector for spam. Stopgap measure
until we figure out something better.
2017-11-29 22:18:05 -08:00
Tim Abbott 6e8f4ffc77 test_signup: Expand tests for invalid confirmation links.
We'll need the expanded test coverage when we move
check_prereg_key_and_redirect to zerver/views/registration.py to avoid
test failures, and these are also tests we should really have anyway.
2017-11-29 22:18:05 -08:00
Tim Abbott be832387c5 user_groups: Fix reporting of empty user groups.
Previously, we weren't doing a proper left join in
user_groups_in_realm_serialized, resulting in empty user groups being
excluded from the query.  We want to leave decisions about excluding
empty user groups to the UI layer, so we include these here.
2017-11-29 16:13:55 -08:00
Tim Abbott b2cb443d24 subs: Fix clearing unread counts when leaving private streams.
Because we use access_stream_by_id here, and that checks for an active
subscription to interact with a private stream, this didn't work.

The correct fix to add an option to active_stream_by_id to accept an
argument indicating whether we need an active subscription; for this
use case, we definitely do not.
2017-11-29 14:40:08 -08:00
Tim Abbott 6bc1a682de notifications: Separate push and email notifications checks.
This is an early step in a larger refactor to properly decouple the
email and push notification code paths.
2017-11-28 17:51:18 -08:00
Sarah ba70b542d3 notification settings: Add stream email notifications setting.
Adds a field to UserProfile for enabling emails about stream
messages and a field to Subscriptions about email
notification preferences.
2017-11-28 17:51:18 -08:00
Greg Price 0ec2a9d259 auth: Try switching to register even if user exists on another realm.
For example, this means that if a user already has an account on one
realm and they try to make an account on another by hitting "Sign in
with Google" (rather than following the little "Register" link to a
"Sign up with Google" button instead), they'll get to make an account
instead of getting an error.

Until very recently, if the user existed on another realm, any attempt
to register with that email address had to fail in the end, so this
logic gave the user a useful error message early.  We introduced it in
c23aaa178 "GitHub: Show error on login page for wrong subdomain"
back in 2016-10 for that purpose.  No longer!  We now support reusing
an email on multiple realms, so we let the user proceed instead.

This function's interface is kind of confusing, but I believe when its
callers use it properly, `invalid_subdomain` should only ever be true
when `user_profile` is None -- in which case the revised
`invalid_subdomain` condition in this commit can never actually fire,
and the `invalid_subdomain` parameter no longer has any effect.  (At
least some unit tests call this function improperly in that respect.)
I've kept this commit to a minimal change, but it would be a good
followup to go through the call sites, verify that, eliminate the use
of `invalid_subdomain`, then remove it from the function entirely.
2017-11-28 16:38:41 -08:00
Vishnu Ks d329636079 tests: Add test for user signup with already-used email.
[Test simplified a bit by greg.]
2017-11-28 16:23:10 -08:00
Greg Price 50e451cd7b populate_db: Create an email-reusing Cordelia in the "lear" realm. 2017-11-28 16:23:10 -08:00
Tim Abbott 586e18b237 auth: Allow accounts with the same email in different realms.
[Modified by greg to (1) keep `USERNAME_FIELD = 'email'`,
(2) silence the corresponding system check, and (3) ban
reusing a system bot's email address, just like we do in
realm creation.]
2017-11-28 16:23:10 -08:00
Tim Abbott e6f460f511 auth: Replace user_email_is_unique validator.
As we migrate to allow reuse of the same email with multiple realms,
we need to replace the old "no email reuse" validators.  Because
stealing the email for a system bot would be problematic, we still ban
doing so.

This commit only affects the realm creation logic, not registering an
account in an existing realm.
2017-11-28 16:23:10 -08:00
Greg Price d6cfa56bc1 test_signup: Test that cloning a system bot's email is not allowed.
Just now this is largely redundant with `test_signup_already_active`;
but very soon when we allow reusing an email across realms, the logic
will diverge.
2017-11-28 16:23:10 -08:00
Greg Price eb8b05b2b5 test_email_mirror: Remove a brittle bit of logic.
This code appears to exist to cover a few extra lines in
zerver/lib/digest.py.  But it's rather brittle, tucked as it is into
the middle of a different test's loop, and with the upcoming
introduction of the `lear` realm in testing, this test code itself
loses coverage.

For now, rather than fix this test code up just delete it; we don't
have 100% coverage on `zerver/lib/digest.py`, while we do on this test
file, so that avoids breaking coverage in CI.  As a followup, we
should add back some logic like this but in a more robust way,
probably as its own separate test method.
2017-11-28 16:23:10 -08:00
Greg Price 453585abb1 test_messages: Clean up some error spew by updating a test.
This test would produce a bunch of log messages with tracebacks,
complaining that `welcome-bot@zulip.com` tried to send cross-realm PMs
and can't.  The issue is that the test overrides
`settings.CROSS_REALM_BOT_EMAILS`, and hasn't kept up with additions
to the normal value for that setting.  Update it so that welcome-bot
is permitted as usual.
2017-11-28 16:23:10 -08:00
Robert Hönig 1e66bd1c02 tests: Add newlines to test_service_bot_system.test_internal_endpoint(). 2017-11-27 21:05:34 -08:00
derAnfaenger c8a5ae753c embedded bots: Consistently use 'storage' instead of 'state.' 2017-11-27 21:05:34 -08:00
Vishnu Ks d4ee3023b2 registration: Require an explicit realm on PreregistrationUser.
This completes the last commit's work to fix CVE-2017-0910, applying
to any invite links already created before the fix was deployed.  With
this change, all new-user registrations must match an explicit realm
in the PreregistrationUser row, except when creating a new realm.

[greg: rewrote commit message]
2017-11-27 14:59:34 -08:00
Vishnu Ks 985768b2fd registration: Check realm against PreregistrationUser realm.
We would allow a user with a valid invitation for one realm to use it
on a different realm instead.  On a server with multiple realms, an
authorized user of one realm could use this (by sending invites to
other email addresses they control) to create accounts on other
realms. (CVE-2017-0910)

With this commit, when sending an invitation, we record the inviting
user's realm on the PreregistrationUser row; and when registering a
user, we check that the PregistrationUser realm matches the realm the
user is trying to register on.  This resolves CVE-2017-0910 for
newly-sent invitations; the next commit completes the fix.

[greg: rewrote commit message]
2017-11-27 14:58:26 -08:00
Rohitt Vashishtha 6734b00c0b markdown: Hide URL if message is only an image.
Hides URL if the message content == image url so that sending gifs or
images feels less cluttered. Uses the url_to_a() function to generate
the expected url string for matching.

Fixes #7324.
2017-11-27 13:30:18 -08:00
Rohitt Vashishtha 3550467899 markdown: Insert text in link-only inline image preview tests.
Appends "Test: " text to some tests to make changes to the image preview
rendering. In the future, if the message is only a link to an image,
the link will be hidden.
2017-11-27 13:30:18 -08:00
Tim Abbott 2210f627a5 signup: Switch active mirror-dummy users to an AssertionError.
Previously, this was a ValidationError, but that doesn't really make
sense, since this condition reflects an actual bug in the code.

Because this happened to be our only test coverage the ValidationError
catch on line 84 of registration.py, we add nocoverage there for now.
2017-11-26 16:13:45 -08:00
Vishnu Ks b9bc1c2b33 Eliminate get_user_profile_by_email from test_classes. 2017-11-26 15:47:56 -08:00
Vishnu Ks 610eb557b8 backend: Make password reset form support multi realm membership. 2017-11-26 15:35:25 -08:00
Vishnu Ks 9f225360dc register: Make /register redirect to find accounts in root domain without realm.
If there is no realm in the root domain don't show
registration form. Instead just redirect to find
accounts page.
2017-11-26 15:32:17 -08:00
Vishnu Ks bab0b913ee test_signup: Extract logic for realm-creation test to a function. 2017-11-26 15:29:16 -08:00
Vishnu Ks 1b5753d021 test_signup: Fix assertIsNone check for realm in RealmCreationTest. 2017-11-26 15:27:33 -08:00
Robert Hönig 0e0a8a2b14 queue processor tests: Call consume by default.
This significantly improves the API for queue_json_publish to not be
overly focused on what the behavior of this function should be in our
unit tests.
2017-11-26 11:45:34 -08:00
Tim Abbott 6968b540c8 mypy: Remove type: ignores that are unnecessary with new mypy.
Mostly these can be removed because the broken LXML stubs were removed
from typeshed in https://github.com/python/typeshed/issues/525.
2017-11-25 10:06:28 -08:00
Tommy Ip e44b8809b2 /team: Render contributors list in tabs client side. 2017-11-22 11:33:24 -08:00
Harshit Bansal 2e7d0cb4cb digests: Fix new stream links.
Fixes: #7479.
2017-11-22 10:07:36 -08:00
rht 3ec90f8b33 zerver/tests: Use python 3 syntax for typing (final). 2017-11-21 22:01:19 -08:00
rht 0260ba82ca zerver/tests: Use python 3 syntax for typing (part 4). 2017-11-21 22:01:19 -08:00
rht 3bf9cd0656 zerver/tests: Use python 3 syntax for typing (part 3). 2017-11-21 22:01:19 -08:00
rht e3daa09b05 zerver/tests: Use python 3 syntax for typing. 2017-11-21 22:01:19 -08:00
rht e169bb0954 zerver: Remove simplejson dependency.
Modified by tabbott to put the third-party code in a new file.

Fixes #6970.
2017-11-21 21:51:48 -08:00
derAnfaenger e526d0c144 embedded bots: Add views to access state. 2017-11-21 21:10:39 -08:00