Commit Graph

7422 Commits

Author SHA1 Message Date
Abijith10 c4420ba0c6 mypy: Use Python 3 type syntax in zerver/webhooks/librato/view.py. 2018-01-23 08:37:56 -08:00
Abijith10 8a0cd1a966 mypy: Use Python 3 type syntax in zerver/webhooks/github_webhook/view.py. 2018-01-23 08:37:56 -08:00
Abijith10 d30c2463c2 mypy: Use Python 3 type syntax in zerver/webhooks/deskdotcom/view.py. 2018-01-23 08:37:56 -08:00
Abijith10 c6804f5c8d mypy: Use Python 3 type syntax in zerver/webhooks/delighted/view.py. 2018-01-23 08:37:56 -08:00
Abijith10 39fe64132d mypy: Use Python 3 type syntax in zerver/webhooks/codeship/view.py. 2018-01-23 08:37:56 -08:00
Abijith10 ead63c475b mypy: Use Python 3 type syntax in zerver/models.py. 2018-01-23 08:37:56 -08:00
Rhea Parekh 11b549e566 slack importer: Add regex for mail links.
`<mailto:foo@foo.com>` is changed to `mailto:foo@foo.com`.
2018-01-23 10:01:15 -05:00
Rhea Parekh df1bb5fd27 slack importer: Fix link regex and add test case. 2018-01-23 10:01:15 -05:00
Rhea Parekh 5b1e9f8181 slack importer: correct the implementation of unzipping slack data file.
Slack zip file unzips in the same folder.
2018-01-23 10:01:15 -05:00
Rhea Parekh 845a8d63bf slack importer: Prevent id clashes for import in an active db.
Previously we had a problem of id clashes while importing converted
slack data into an existing zulip instance with realms which are actively
populating the database.

This counts the total objects to be imported and does a db transaction
to increase the SEQUENCE number for that table by that number,
and hence allocates a range of ids for the to be converted slack data
objects.
2018-01-23 10:01:15 -05:00
Weronika Grzybowska 7ac7100a1d messages: Make checking for status message consistent with backend.
Adds a check for newline that was present on backend, but missing in the
frontend markdown implementation. Updating messages uses is_me_message flag
received from server instead of its own partial test. Similarly, rendering
previews uses markdown code.

Fixes #6493.
2018-01-23 09:26:41 -05:00
Robert Hönig 9a15c4e3ff Add bot services to page_params.
This is the first step for allowing users
to edit a bot's service entries, name the
outgoing webhook configuration entries. The
chosen data structures allow for a future
with multiple services per bot; right now,
only one service per bot is supported.
2018-01-23 07:29:00 -05:00
Robert Hönig 06fb868482 Send an event to bot owners when a bot is created. 2018-01-23 07:29:00 -05:00
Vishnu Ks 036dc53d20 messages: Rename last_visible_message_id to first_visible_message_id. 2018-01-22 19:53:44 -08:00
Eeshan Garg 00d3f19c0a integrations/email: Render settings.EMAIL_GATEWAY_EXAMPLE correctly. 2018-01-22 18:05:20 -05:00
YJDave 8905744bb1 stream settings: Add period at the end of error messages. 2018-01-19 13:05:43 -05:00
Aditya Bansal f272ea9087 scheduledmessages: Start using/expecting delivery_type as a param. 2018-01-19 11:33:11 -05:00
Aditya Bansal dcce8e7219 scheduledmessages: Add delivery_type field to the table. 2018-01-19 11:33:11 -05:00
Aditya Bansal c770bdaa3a reminder_bot: Add infra for adding reminder bot to every realm. 2018-01-19 11:33:11 -05:00
Aditya Bansal f46d098558 schedulemessages: Add handle_deferred_message() to handle requests.
This is responsible for:
1.) Handling all the incoming requests at the
messages endpoint which have defer param set. This is similar to
send_message_backend apart from the fact that instead of really
sending a message it schedules one to be sent later on.
2.) Does some preliminary checks such as validating timestamp for
scheduling a message, prevent scheduling a message in past, ensure
correct format of message to be scheduled.
3.) Extracts time of scheduled delivery from message.
4.) Add tests for the newly introduced function.
5.) timezone: Add get_timezone() to obtain tz object from string.
This helps in obtaining a timezone (tz) object from a timezone
specified as a string. This string needs to be a pytz lib defined
timezone string which we use to specify local timezones of the
users.
2018-01-19 11:33:11 -05:00
rht ee06da9350 bugdown: Linkify Bitcoin URI. 2018-01-19 06:07:43 -05:00
Umair Khan ea8f1efb63 tests: Re-raise exception if not import error. 2018-01-19 06:02:05 -05:00
Umair Khan 8f316183e0 tests: Make import error tracebacks terse.
This will also disable exception chaining; we do not need it here.
2018-01-19 06:02:05 -05:00
Vishnu Ks 7dbb4cdae2 uploads: Remove ExceededQuotaError class.
We are no longer using it.
2018-01-18 15:38:08 -08:00
Tim Abbott 2c16c76551 beeminder: Fix buggy coding tag at top of file. 2018-01-18 09:07:06 -08:00
Umair Khan a994dba583 tests: Give more descriptive import error.
This code takes care of the environment running Python 3.4 when
test label is passed directly to the test-backend command:
./tools/test-backend test_alert_words
2018-01-18 07:13:26 -05:00
Umair Khan 0eda941822 tests: Verify imports in test modules. 2018-01-18 07:13:26 -05:00
Umair Khan 7248a3366b tests: Change print_error_message to check_import_error. 2018-01-18 07:13:26 -05:00
Umair Khan f3c86c9e58 tests: Remove old import testing code.
We will add the modified version of this code in the next commit.
2018-01-18 07:13:26 -05:00
Umair Khan c26cdee95d get_test_names: Update to accept ParallelTestSuite arg. 2018-01-18 07:13:26 -05:00
Alena Volkova aa58e1fcda tests: Make import failure error messages more consistent. 2018-01-18 07:13:26 -05:00
Alena Volkova 13b1334fe4 tests: Refactor error handling of import failures. 2018-01-18 07:13:26 -05:00
Greg Price 3d423a3c7b logging: Fix corrupted default format for our logfiles.
This was changed by accident when the syntax was rearranged,
in 3f4bf2d22 "zerver/lib: Use python 3 syntax for typing"
back in November.
2018-01-17 18:10:17 -08:00
Shubham Dhama 797fa62fde test auth.py: Add tests for json_fetch_api_key function. 2018-01-17 15:15:51 -05:00
infinitelooped 252a485621 integrations: Add documentation for Groove. 2018-01-17 13:00:11 -05:00
Privisus e5fc49e23a integrations: Add documentation for Raygun. 2018-01-17 12:53:22 -05:00
Privisus c758636c1e integrations: Add webhook code, API endpoint, and tests for Raygun. 2018-01-17 12:53:22 -05:00
Rhea Parekh 7153d20205 slack importer: append Message id from the db. 2018-01-17 11:46:27 -05:00
Rhea Parekh 58f1f0e13e slack importer: append Recipient id from the db.
Recipient id should be updated from the database,
otherwise there will be foreign key clashes.
2018-01-17 11:46:27 -05:00
Rhea Parekh 138aba3d89 slack importer: Add unit tests for slack message conversions. 2018-01-17 09:38:53 -05:00
Rhea Parekh c86411d9a2 slack importer: Move message conversions to a new module. 2018-01-17 09:38:53 -05:00
Rhea Parekh c1d336f5d1 slack importer: Refactor checking for availability of realm subdomain. 2018-01-17 09:38:53 -05:00
Ricky a90e99dd3e webhooks: Add Beeminder webhook. 2018-01-17 09:24:12 -05:00
fredfishgames 4cb9edecda integrations: Add documentation for Opbeat. 2018-01-17 09:17:19 -05:00
fredfishgames 4e2b99a98b integrations: Add webhook code, API endpoint, and tests for Opbeat. 2018-01-17 09:17:19 -05:00
Shreyansh Dwivedi b0fb7aa6b2 bugdown: Add preview for vimeo videos.
This also amends a commit from Brock Whittaker <brock@zulipchat.com>
that merges two separate functions for YouTube videos and Vimeo videos
into a generic video recall function.

Fixes #7550.
2018-01-17 09:11:43 -05:00
Eeshan Garg 457c5a7735 webhooks/gci: Support 'outoftime' event.
This commit adds support for the 'outoftime' event. Such an event
is triggered when the task deadline for a particular task has
passed.
2018-01-16 15:57:35 -08:00
Viraat Chandra 64034241a0 webhooks: Create Intercom integration and docs. 2018-01-16 08:51:17 -05:00
Xavier Cooney 8c5e74afe9 test_messages.py: Remove unecessary imports. 2018-01-16 08:16:43 -05:00
Xavier Cooney b0f501ff6c test_helpers.py: Remove unecessary imports. 2018-01-16 08:16:43 -05:00
Xavier Cooney 0c48f5bf09 test_classes.py: Remove unecssary imports. 2018-01-16 08:16:43 -05:00
XavierCooney 35dc203d58 queue_processors.py: Remove unecessary imports. 2018-01-16 08:16:43 -05:00
Robert Hönig 93f47fa700 outgoing webhooks: Rename "Base URL" to "Endpoint URL".
Based on user feedback. "Endpoint URL" gives a clearer
idea of the field's purpose.
2018-01-16 08:11:51 -05:00
Sivagiri Visakan aeeb9d07a8 integrations: Handle errors in Groove payloads. 2018-01-16 07:31:50 -05:00
Eeshan Garg a14b52ef1d api docs: Document the `DELETE /users/me/subscriptions` endpoint.
Note that there is currently no JavaScript method in zulip-js
to call this endpoint.
2018-01-12 07:30:38 -05:00
Eeshan Garg 8c723e8c27 api docs: Document the POST /api/v1/users endpoint.
Note that there is no JavaScript equivalent for requesting this
endpoint in zulip-js yet.
2018-01-12 07:30:38 -05:00
Kiy4h 68d86b62fa integrations: Add webhook payloads for rollbar. 2018-01-11 06:32:37 -05:00
Xavier Cooney e417a99830 webhooks/gitlab: Support topic parameter to specify a topic.
Fixes #7743.
2018-01-11 06:29:55 -05:00
Aditya Bansal ec1297c1e8 schedulemessages: Add delivery system for scheduled message. 2018-01-10 09:18:02 -05:00
Aditya Bansal c64766e55e scheduledmessages: Add funcs to verify and schedule messages.
We add two functions:

1.) check_schedule_message(): This function is responsible for
doing the essential initial checkes to verify the validity of
the message. These checkes include things like if user is
allowed to send messages to some stream or not or if the user is
a super_user. All this is basically done by further calling
check_message() with appropriate parameters. This is on the same
lines as is check_send_message().

2.) do_schedule_messages(): This function is responsible for
creating ScheduleMessage table rows for a list of messages that
are to be scheduled. This basically accumulates the ScheduleMessage
objects in a list and then bulk creates the rows.
2018-01-10 09:18:02 -05:00
Aditya Bansal 6c36cf3c82 scheduledmessages: Add model for the ScheduledMessage table. 2018-01-10 09:18:02 -05:00
Andy Perez c209069fcd bugdown: Improve image inlining logic.
Fix #7537
2018-01-09 16:43:22 -05:00
Robert Hönig d1d8365a6b embedded bots: Add config data UI.
This adds UI fields in the bot settings for specifying
configuration values like API keys for a bot. The names
and placeholder values for each bot's config fields are
fetched from the bot's <bot>.conf template file in the
zulip_bots package. This also adds giphy and followup
as embedded bots.
2018-01-09 11:50:54 -05:00
infinitelooped ffa7637215 integrations: Add webhook code, API endpoint, and tests for Groove. 2018-01-09 09:32:52 -05:00
Alena Volkova 9d1063d362 urls: Move the json/fetch_api_key endpoint to be an API-style route. 2018-01-08 13:15:52 -05:00
Alena Volkova 7c22045158 test_decorators: Replace the json/fetch_api_key endpoint in tests.
This endpoint is about to become an API-style route and have the legacy
decorator removed from its view. Other endpoints will be used in tests
instead of it.
2018-01-08 13:15:52 -05:00
Rhea Parekh b583f1f0ff slack importer: Add regex for markdown mapping.
* For strikethrough formatting: Slack's '~strike~' to Zulip's '~~strike~~'.
    * For bold formatting: Slack's '*bold*' to Zulip's '**bold**'.
    * For italic formatting: Slack's '_italic_' to Zulip's '*italic*'.
    * For mentioning formatting: Slack's <@slack_id|short_name> to Zulip's @**full_name**.
    * Checking links.
2018-01-08 08:05:50 -05:00
Rhea Parekh a6e0bcd5f6 Slack importer: Map messages and usermessages. 2018-01-08 08:05:50 -05:00
Rhea Parekh 24a86a524b slack importer: Quiet the unzip command. 2018-01-08 08:05:50 -05:00
Rhea Parekh 4019a456d1 slack importer: Rename messages file.
Rename messages file from '/message-000001.json' to
'/messages-000001.json'
2018-01-08 08:05:50 -05:00
Rhea Parekh 400c968b0e slack importer: Remove unnecessary comment. 2018-01-08 08:05:50 -05:00
Rhea Parekh c875428a83 slack importer: Extract get_user_full_name() helper. 2018-01-08 08:05:50 -05:00
Robert Hönig 3420ed1fcc validator.py: Cover missing lines in check_dict(). 2018-01-07 15:23:17 -05:00
Eeshan Garg a01f8c088a api docs: Document the POST /api/v1/users/me/subscriptions endpoint.
There is a JavaScript equivalent for this endpoint but the npm
package has not yet been released.
2018-01-07 15:19:51 -05:00
Eeshan Garg ae59b08316 api docs: Document the GET /api/v1/users/me endpoint.
Note that there is no JavaScript equivalent for requesting this
endpoint in zulip-js yet.
2018-01-07 15:19:51 -05:00
Eeshan Garg 045aacbc67 api docs: Document the PATCH /api/v1/messages/<msg_id> endpoint.
Note that there is no JavaScript equivalent for requesting this
endpoint in zulip-js yet.
2018-01-07 15:19:51 -05:00
Eeshan Garg 6f01b042a1 api docs: Document the DELETE /api/v1/events endpoint.
Note that there is no JavaScript equivalent for requesting this
endpoint in zulip-js yet.
2018-01-07 15:19:51 -05:00
Robert Hönig c745d1901c bot_config.py: Add function for loading config file templates.
load_bot_config_template(bot) parses the <bot>.conf
template file, which can be found in the zulip_bots
package for each bot. It then returns the INI content
of that file as a dict.
2018-01-07 20:05:52 +01:00
Robert Hönig ed7208fd6e Make EmbeddedBotHandler.get_config_info comply with ExternalBotHandler. 2018-01-07 20:05:52 +01:00
Robert Hönig 9cc059f4aa bot_config.py: Raise exception when getting config data from bot without config entries. 2018-01-07 20:05:52 +01:00
Robert Hönig 83d0c7be31 embedded bots: Run bot.initialize() if bot has this function. 2018-01-07 20:05:52 +01:00
Robert Hönig dcd62f92fb validator.py: Extend functionality of check_dict().
With this commit, check_dict() can be used to verify
that all keys and/or values of a dict are of a certain
type.
2018-01-07 20:04:52 +01:00
Vishnu Ks 5cadc6365a slack importer: Extract data into directory of zip file.
Currently the zip file is extracted to the root of
zulip directory no matter where the the zip file.
The extracted data is not useful after running the command
which pollutes the zulip directory. It make more sense to
extract it to the same directory of zip file especially
when the zip file gets downloaded to /temp like in the tests.
2018-01-07 09:50:27 -05:00
Robert Hönig 29a343cb44 Rename BotUserConfigData to BotConfigData.
This is in compliance with the naming of
BotStorageData.
2018-01-06 14:13:18 -05:00
Alena Volkova 5bbc46762f tests: Improve error handling of import failures.
Fixes #6776.
2018-01-05 12:03:37 -05:00
Privisus 4f700ddcbf integrations: Add webhook payloads for Raygun. 2018-01-04 11:44:39 -05:00
Eeshan Garg 479bc41ca5 api docs: Remove old and outdated endpoint docs. 2018-01-04 10:17:29 -05:00
Eeshan Garg 1fd71287d6 api docs: Document the GET /api/v1/events endpoint. 2018-01-04 10:17:29 -05:00
Vishnu Ks e6d3f8895f messages: Create function to calculate first visible message id. 2018-01-04 08:52:40 -05:00
Vishnu Ks f2a7d44c29 models: Add message_visibility_limit attribute to Realm. 2018-01-04 08:52:40 -05:00
MadElf1337 7c113d185f integrations: Add webhook payloads for Facebook. 2018-01-03 14:47:50 -05:00
Eeshan Garg f620525373 api docs: Document the POST /api/v1/register endpoint. 2018-01-03 12:23:30 -05:00
Alena Volkova 45f0c76c44 settings: Limit the creation of generic bots.
This commit adds a setting to limit creation of generic bots
to admins for realms that want that restriction.  (Generic
bots, apart from being considered spammy on some realms,
have less locked down permissions than webhook bots).

Fixes #7066.
2018-01-02 18:12:22 -05:00
Eeshan Garg 18c3bec667 webhooks/gci: Support "unassign" event type.
This commit adds support for the "unassign" event type, the
payloads for which are generated when a mentor unassigns a student
from a task.
2018-01-02 16:42:00 -05:00
Eeshan Garg 2c7f4d2187 webhooks/gci: Support "extend" event type.
This commit adds support for events of type "extend", the payloads
for which are generated when a mentor extends the deadline for a
task.
2018-01-02 16:42:00 -05:00
Aastha Gupta daf86eb664 settings: Add "text" option to emoji_set model.
We no longer have a special UI setting and model
field ("emoji_alt_code") for saying users want text-only
emojis.  We now instead make "text" be a fifth choice
for "emojiset".

Fixes #7406
2018-01-02 14:55:01 -05:00
Vishnu Ks 277d463d23 messages: Add option to limit users message history. 2018-01-02 13:45:01 -05:00
MadElf1337 e7f1a9ee34 inetgrations: Add webhook payloads for Honeybadger. 2018-01-02 11:45:33 -05:00
YJDave fa44d2ea69 settings: Remove autoscroll_forever setting.
Fixes #6845
2018-01-02 10:35:49 -05:00
Eeshan Garg 80405747ce test_docs: Test new API docs (templates/zerver/api/*). 2017-12-30 11:03:23 -05:00
rht 185fd99816 mypy: Use Python 3 type syntax in several files. 2017-12-30 07:34:51 -05:00
rht 1324d9f000 mypy: Use Python 3 type syntax in zerver/views. 2017-12-30 07:34:05 -05:00
rht 2e8adbde12 mypy: Use Python 3 type syntax in zerver/lib/actions.py. 2017-12-30 07:33:07 -05:00
Eeshan Garg 89f622937d api docs: Generate Arguments table from JSON.
This commit does the following:
* Move the Arguments table data from stream-message.md and
  private-message.md to a JSON file.
* Add a Markdown extension that allows one to include and render
  a table from a JSON file like so:
    {generate_arguments_table|arguments.json|private-stream.md}
* Use Bootstrap's .table class to format the table instead of
  relying on custom CSS.
2017-12-29 17:03:36 -05:00
Eeshan Garg 94bc9d52d1 api docs: Split usage.md into two separate docs.
This commit splits usage.md into two separate docs,
stream-message.md and private-message.md. The arguments and return
values for sending a stream message are somewhat different from
those of sending a private message, so it made sense to split the
two up for clarity.
2017-12-29 17:03:36 -05:00
YJDave 29905ad5d6 create stream: Before sending notificaion, check if NOTIFICATION_BOT exists.
There might be case that NOTIFICATION_BOT is none, so before sending stream
announce notification, check first if settings.NOTIFICATION_BOT is not none.
2017-12-29 08:59:55 -05:00
MadElf1337 39fc26deff integrations: Add webhook payloads for Runscope. 2017-12-29 07:27:20 -05:00
akashnimare ef51a39552 markdown: Update regex for strikethrough.
This uses the correct regex for strikethrough. Also, added
a test to make sure that strikethrough works when it contains
link with whitespace.

Fixes #7596.
2017-12-28 15:59:04 -05:00
MadElf1337 de62e11c23 integrations: Add webhook payloads for StatusPage.io. 2017-12-28 15:53:06 -05:00
Steve Howell 80477da481 quick fix: Stop tracebacks with missedmessage_hook.
A `None` value is not properly handled in this function, which
indicates some lack of testing or a recent regression we don't
understand.  We were getting lots of tracebacks from this line
of code on our test server:

    mentioned = 'mentioned' in flags and 'read' not in flags
2017-12-28 08:10:56 -05:00
Kiy4h 2e35b7fd60 mypy: Use Python 3 type syntax in zerver/webhooks/taiga/view.py. 2017-12-28 07:26:44 -05:00
Kiy4h 93cc19e3f8 mypy: Use Python 3 type syntax in zerver/webhooks/helloworld/view.py. 2017-12-28 07:26:44 -05:00
Kiy4h 3b7fbc1a1d mypy: Use Python 3 type syntax in zerver/webhooks/gosquared/view.py. 2017-12-28 07:26:44 -05:00
Kiy4h 6fa51be822 mypy: Use Python 3 type syntax in zerver/views/push_notifications.py. 2017-12-28 07:26:44 -05:00
Kiy4h 64413e3b91 mypy: Use Python 3 type syntax in zerver/views/email_mirror.py. 2017-12-28 07:26:44 -05:00
Kiy4h 238cd2b132 mypy: Use Python 3 type syntax in zerver/views/alert_words.py. 2017-12-28 07:26:44 -05:00
Kiy4h 78e71faefe mypy: Use Python 3 type syntax in zerver/lib/test_fixtures.py. 2017-12-28 07:26:44 -05:00
Rhea Parekh ed4054d110 Import script: Check and add system bots after every import.
This checks for the existing system bots and adds them if they
aren't included in the import.
2017-12-27 07:52:45 -05:00
Rhea Parekh 62aec0688f Slack importer: Add slack to zulip data conversion script as a manage.py command. 2017-12-27 07:50:53 -05:00
Rhea Parekh cd332a7f59 Slack importer: Append ids to the existing ids in the database.
For import with the condition `--import-into-nonempty`, the ids
of the mapped data will be appended to the existing ids in the
database.
2017-12-27 07:50:53 -05:00
Rhea Parekh 5b37c5562b Slack importer: Add the slack to zulip data conversion files. 2017-12-27 07:50:53 -05:00
Rhea Parekh 3d4ee3b862 zerver/tests/test_email_mirror: Sweep force_str. 2017-12-26 09:09:31 -05:00
Rhea Parekh 25e5969add zerver/lib/mobile_auth_otp: Sweep force_str. 2017-12-26 09:09:31 -05:00
Rhea Parekh cc872ad976 zerver/lib/upload.py: Sweep force_bytes from ensure_medium_avatar_image. 2017-12-26 09:09:31 -05:00
Rhea Parekh 092472f42d tests: Add test coverage for uploading avatars to S3UploadBackend.
Add test coverage for the function upload_avatar_image and
ensure_medium_avatar_image in S3UploadBackend.
2017-12-26 09:09:31 -05:00
Rhea Parekh bebd8df728 zerver/webhooks/github/view: Sweep force_str. 2017-12-26 09:09:31 -05:00
Rhea Parekh 99cb001345 zerver/webhooks/beanstalk/view.py: Sweep force_bytes and force_str. 2017-12-26 09:09:31 -05:00
Rhea Parekh b792a90842 Remove unused imports in the codebase. 2017-12-26 09:09:31 -05:00
derAnfaenger 94c8e8b8e7 embedded bots: Strip @-mention from message.
This is in order to comply with the most recent
code in the `zulip_bots` package.
2017-12-26 08:50:00 -05:00
Xavier Cooney f66f04f83a mypy: Use Python 3 type syntax in 'bitbucket/tests.py'. 2017-12-26 08:34:38 -05:00
Xavier Cooney 63142385ca mypy: Use Python 3 type syntax in 'send_password_reset_email.py'. 2017-12-26 08:34:38 -05:00
Xavier Cooney 34aee15d63 mypy: Use Python 3 type syntax in 'gitlab/view.py'. 2017-12-26 08:34:38 -05:00
Xavier Cooney 36b58ff6fc mypy: Use Python 3 type syntax in 'views/users.py'. 2017-12-26 08:34:38 -05:00
Xavier Cooney 78042d7f18 mypy: Use Python 3 type syntax in 'webhooks/gogs/tests.py'. 2017-12-26 08:34:38 -05:00
nyan-salmon 51be0e5e76 mypy: Use Python 3 type syntax in zerver/webhooks/solano/view.py. 2017-12-26 08:34:06 -05:00
nyan-salmon 0fd0332a2c mypy: Use Python 3 type syntax in zerver/views/custom_profile_fields.py. 2017-12-26 08:34:06 -05:00
nyan-salmon 613856c33b mypy: Use Python 3 type syntax in zerver/lib/send_email.py. 2017-12-26 08:34:06 -05:00
nyan-salmon 4f4d689f86 mypy: Use Python 3 type syntax in zerver/webhooks/splunk/view.py. 2017-12-26 08:34:06 -05:00
nyan-salmon 6c5f66c561 mypy: Use Python 3 type syntax in zerver/webhooks/slack/view.py. 2017-12-26 08:34:06 -05:00
Viraat Chandra de3eb89d5c mypy: Use Python 3 syntax for typing in `zerver/lib/request.py`. 2017-12-26 08:31:43 -05:00
Viraat Chandra bbf24ec68e mypy: Use Python 3 syntax for typing in `zerver/lib/webhooks/git.py`. 2017-12-26 08:31:43 -05:00
Viraat Chandra 553711e671 mypy: Use Python 3 syntax for typing in `zerver/tests/test_home.py`. 2017-12-26 08:31:43 -05:00
Viraat Chandra 0494902e6d mypy: Use Python 3 syntax for typing in `zerver/views/auth.py`. 2017-12-26 08:31:43 -05:00
fredfishgames 55801d2feb mypy: Use Python 3 type syntax in zerver/tornado/event_queue.py. 2017-12-26 08:31:09 -05:00
fredfishgames ccc3d69f61 mypy: Use Python 3 type syntax in zerver/webhooks/bitbucket2/tests.py. 2017-12-26 08:31:09 -05:00
greysome 6bcc01bfee mypy: Use Python 3 type syntax in zerver/webhooks/bitbucket/view.py 2017-12-26 08:30:33 -05:00
greysome f2f9efd1a7 mypy: Use Python 3 type syntax in zerver/lib/integrations.py 2017-12-26 08:30:33 -05:00
greysome fb7ee942c4 mypy: Use Python 3 type syntax in zerver/lib/export.py 2017-12-26 08:30:33 -05:00
greysome da9c808db4 mypy: Use Python 3 type syntax in zerver/webhooks/transifex/view.py 2017-12-26 08:30:33 -05:00
greysome 64ce353d4b mypy: Use Python 3 type syntax in zerver/webhooks/circleci/view.py 2017-12-26 08:30:33 -05:00
greysome bbe2d91d31 mypy: Use Python 3 type syntax in zerver/forms.py 2017-12-26 08:30:33 -05:00
Xavier Cooney 06b462ba19 mypy: Use Python 3 type syntax in tests/test_auth_backends.py. 2017-12-26 08:29:30 -05:00
Xavier Cooney 61704dc20b mypy: Use Python 3 type syntax in views/pointer.py. 2017-12-26 08:29:30 -05:00
Xavier Cooney 5b2819645e mypy: Use Python 3 type syntax in decorator.py. 2017-12-26 08:29:30 -05:00
Rishi Gupta e62d9742aa emails: Fix notify_new_login emails have unserializable context.
Regression introduced in e0f30d0 (PR #7569).
2017-12-22 16:34:18 -05:00
Steve Howell c33102f692 Fix mypy error in AdminNotifyHandler.emit(). 2017-12-22 15:02:18 -05:00
Greg Price ac6f3373b6 errors tests: Deduplicate and isolate the magical part.
This makes the tests look a lot more boring and straightforward, in
addition to being a bit shorter.
2017-12-22 11:52:13 -05:00
Greg Price de5c944980 errors tests: De-indent a bunch of code.
This diff is nothing but dedentation -- it's empty under
`git diff -b`.  These with-statements are only needed for
a pretty narrow scope of code, so make that clear in the
source.
2017-12-22 11:52:13 -05:00
Greg Price ca5c991994 errors: Simplify manual testing of error emails.
There are two different things you need to patch in order to get error
emails (at `/emails`) in dev.  Flip one of them in dev all the time,
and make the comment on the other a bit more explicit.
2017-12-22 11:52:13 -05:00
Greg Price a34c01780f errors: Compute deployment metadata on the right deployment.
When I added this "Deployed code" feature to the error reporting,
I apparently hadn't worked out enough of how this code works to
realize that `notify_server_error` may be in a different process,
at a different time and potentially even on a different machine
from the actual error being reported.

Given that architecture, all the data about the error must be computed
in `AdminNotifyHandler`, before sending the report through the queue,
or else it risks being wrong.  The job of `notify_server_error` and
friends is only to format the data and send it off.  So, move the
implementation of this feature in order to do that.

(@showell added some "nocoverage" directives here for code that
is hard to test (exceptions being thrown, deployment files not
existing) and that was originally part of a file that didn't
require 100% coverage)
2017-12-22 11:52:13 -05:00
Greg Price eea1ceb0db errors: Show what logger and source code the message comes from. 2017-12-22 11:52:13 -05:00
Greg Price 77e47900dd errors: Stop showing a long list of Nones when there's no request.
This had contributed to making the emails look broken and of uncertain
accuracy.
2017-12-22 11:52:13 -05:00
Greg Price 68fa29c3ec errors: Make Zulip and email paths more parallel.
This helps prevent them from diverging and getting different sets of
features and fixes.  As a bonus, the email path gets a nice tweak that
the Zulip path has had for years, since f7f2ec0ac, which makes the
emails clearer and less broken-looking when logging a message with no
stack trace.
2017-12-22 11:52:13 -05:00
Greg Price 8c0fd5beaf errors: Clean up the logic slightly.
This deduplicates a little bit of logic, and also has us always put
things into `report` the same way.

Empirically an exception in this codepath is very rare, so we won't
complicate the code by trying to salvage a lot of partial information
if it happens -- just log the traceback, and try to get a minimal
notification sent of the bare fact this happened.
2017-12-22 11:52:13 -05:00
Greg Price b15231dfc2 logging: Rename AdminZulipHandler to AdminNotifyHandler.
This name hasn't been right since f7f2ec0ac back in 2013; this handler
sends the log record to a queue, whose consumer will not only maybe
send a Zulip message but definitely send an email.  I found this
pretty confusing when I first worked on this logging code and was
looking for how exception emails got sent; so now that I see exactly
what's actually happening here, fix it.
2017-12-22 11:52:13 -05:00
neiljp (Neil Pilgrim) 73a834990b mypy: Swap order of `user` initialization in ZulipPasswordResetForm.save.
When running with `--strict-optional`, this helps mypy see what's
going on here.

[Fix changed by greg.]
2017-12-23 01:22:22 +09:00
neiljp (Neil Pilgrim) 7c819f4d90 mypy: Account for Optional variables in addressee.py. 2017-12-23 01:22:22 +09:00
neiljp (Neil Pilgrim) 57e7ec94a0 mypy: Adjust 2 emoji.py functions to allow for strict-optional. 2017-12-23 01:22:22 +09:00
neiljp (Neil Pilgrim) 451af39561 mypy: Correct get*notifications_stream to return Optional[Stream]. 2017-12-23 01:22:22 +09:00
neiljp (Neil Pilgrim) d01dcbd018 mypy: Amend tos_version parameter of create_user_profile to be Optional.
tos_version is marked as Optional[Text] in UserProfile & is Optional
in other places within zerver/lib/create_user.py
2017-12-23 01:22:22 +09:00
Aastha Gupta 78e8d18ef3 integrations: Rename HUBOT_INTEGRATIONS_LEGACY.
It is now simply called HUBOT_INTEGRATIONS.

Fixes #7402
2017-12-22 10:38:47 -05:00
Angelika Serwa 2f575cca72 Add Dropbox webhook integration.
This is just a basic Dropbox webhook integration. It just
notifies a user when something has changed, it does not
specify what changed. Doing so would require storing data,
as Dropbox API was created mainly for file managers, not
integrations like this.
Closes #5672
2017-12-22 10:36:21 -05:00
Marco Burstein 6fc50fa21a mypy: Use Python 3 type syntax in `zerver/webhooks/sentry/view.py`. 2017-12-22 08:17:00 -05:00
Marco Burstein 34001081c2 mypy: Use Python 3 type syntax in `zerver/webhooks/yo/view.py`. 2017-12-22 08:17:00 -05:00
Marco Burstein 678b43494b mypy: Use Python 3 type syntax in `zerver/webhooks/airbrake/view.py`. 2017-12-22 08:17:00 -05:00
Marco Burstein 9327048f6f mypy: Use Python 3 type syntax in `zerver/lib/events.py`. 2017-12-22 08:17:00 -05:00
Marco Burstein 2acf3cf1d4 mypy: Use Python 3 type syntax in `zerver/webhooks/mention/view.py`. 2017-12-22 08:17:00 -05:00
Marco Burstein 017476cdfb mypy: Use Python 3 type syntax in `zerver/lib/create_user.py`. 2017-12-22 08:17:00 -05:00
Marco Burstein e7f0139f69 mypy: Use Python 3 type syntax in `zerver/webhooks/travis/view.py`. 2017-12-22 08:17:00 -05:00
Tommy Ip 9ea7b05cfe bugdown: Rewrite markdown link to relative link conditionally. 2017-12-22 08:10:42 -05:00
Tommy Ip efe2d497f4 bugdown: Avoid matching uri with regex. 2017-12-22 08:10:42 -05:00
Tommy Ip 9ce797d68c bugdown: Extract rewrite_if_relative_link function. 2017-12-22 08:10:42 -05:00
Umair Khan 898acc4278 missed-message-worker: Send email through email_senders.
We have shifted to a generic queue to send all the emails. This queue
can retry in case of network issues; this makes sure that the emails are
always sent.
2017-12-20 19:36:27 -08:00
Umair Khan e0f30d02f2 email-worker: Send login email through queue.
Fixes #7460
2017-12-20 19:36:27 -08:00
Umair Khan 68513952fb email-worker: Create EmailSendingWorker.
This commit just copies all the code from MissedMessageSendingWorker
class to a new EmailSendingWorker class. All the logic to send an email
through a queue was already there. This commit only makes the logic
generic. It does so by creating a special purpose queue called
'email_senders' to send any type of email. To make
MissedMessageSendingWorker still work we derive it from
EmailSendingWorker. All the tests that were testing
MissedMessageSendingWorker now run against EmailSendingWorker.
2017-12-20 19:36:27 -08:00
Vishnu Ks 16d8244c0a tests: Eliminate 'Sending invitation' output spam in test_signup.
Fixes #7563
2017-12-20 18:50:01 -08:00
neiljp (Neil Pilgrim) 25d5a2ee1c requirements: Upgrade mypy to 0.560.
Fixes #7835.
2017-12-20 18:09:36 -08:00
Rishi Gupta c2a41c268f models: Update Realm.get_admin_users to not use select_related().
None of the uses of this function in the codebase follow any of the foreign
keys of UserProfile.
2017-12-20 08:51:58 -05:00
snlkapil 1589c597d2 settings: Show local time instead of UTC in invitations tab. 2017-12-20 07:08:46 -05:00
Rishi Gupta 27a8ef86e4 actions: Add realm deactivation and reactivation to RealmAuditLog. 2017-12-19 17:46:36 -08:00
Rishi Gupta 4261307e27 emails: Clear ScheduledEmail emails on realm deactivation.
This only affects the invitation reminder emails; clear_scheduled_emails was
already clearing the other two types.
2017-12-19 17:46:36 -08:00
Rishi Gupta 869b4d41ef models: Add ScheduledEmail.realm.
The two extra queries in the test are due to the assert in
send_future_email.
2017-12-19 17:46:36 -08:00
Rishi Gupta 87e164043f portico: Rename plans at /plans. 2017-12-19 15:36:13 -08:00
ViRu-ThE-ViRuS c8aadaa408 integrations: Add webhook payloads for Intercom. 2017-12-19 16:50:06 -05:00
Eeshan Garg 8936d03f3a webhooks/gitlab: Support Merge Request Hook approved event type.
Such payloads are generated when a GitLab repository has merge
request approvals enabled and a project member approves a merge
request. Approving is not the same as merging.
2017-12-18 19:12:54 -05:00
Andy Perez 44e6d9b54b testing: Add nice diffs to markdown python tests. 2017-12-18 19:03:38 -05:00
Andy Perez 695affd44e node tests: Compare markdown using semantic equivalence.
Fix #4367
2017-12-18 19:03:38 -05:00
Greg Price 776af2e248 push notifs: Simplify content-truncation code slightly.
This should be a pure refactor.
2017-12-18 15:32:56 -08:00
Greg Price 014900c2e5 push notifs: Include same stream and sender info on GCM as on APNs.
This takes the information added to APNs payloads by #7080, and adds
to our GCM payloads the parts that it didn't already have.
2017-12-18 15:32:56 -08:00
Greg Price e077d6fc74 push notifs: Factor out common payload fields for APNs and GCM.
This should be a pure refactor, producing exactly the same payloads.
2017-12-18 15:32:56 -08:00
Eeshan Garg 21008a49b9 webhooks/gci: Support approve-pending-pc event type.
This event is generated when a mentor approves a task but
GCI is still waiting on a student's parental consent.
2017-12-18 10:00:20 -08:00
Callum Fraser a61a7f3965 mypy: Use Python 3 type syntax in zerver/webhooks/pingdom/view.py. 2017-12-18 11:32:52 -05:00
Callum Fraser 4508264f25 mypy: Use Python 3 type syntax in zerver/webhooks/opsgenie/view.py. 2017-12-18 11:32:52 -05:00
Callum Fraser 52f0c4472e mypy: Use Python 3 type syntax in zerver/webhooks/ifttt/view.py. 2017-12-18 11:32:52 -05:00
Callum Fraser 23b29212a3 mypy: Use Python 3 type syntax in zerver/webhooks/greenhouse/view.py. 2017-12-18 11:32:52 -05:00
Callum Fraser 403ea486e4 mypy: Use Python 3 type syntax in zerver/webhooks/gogs/view.py. 2017-12-18 11:32:52 -05:00
Callum Fraser 33e1879295 mypy: Use Python 3 type syntax in zerver/webhooks/beanstalk/view.py. 2017-12-18 11:32:52 -05:00
Callum Fraser c55685063f mypy: Use Python 3 type syntax in zerver/webhooks/appfollow/view.py. 2017-12-18 11:32:52 -05:00
Callum Fraser b0c5eb5482 mypy: Use Python 3 type syntax in zerver/views/typing.py. 2017-12-18 11:32:52 -05:00
Callum Fraser 0438da5c16 mypy: Use Python 3 type syntax in zerver/lib/avatar.py. 2017-12-18 11:32:52 -05:00
civilnewsnetwork 0ce8b69fd6 Renamed 'Register' to 'Sign up' including translations 2017-12-18 10:02:15 -05:00
Alena Volkova 3cfb2d57b2 tests: Refactor the method for encoding credentials. 2017-12-18 09:24:09 -05:00
Alena Volkova 39915d9835 tests: Replace all get_auth usages with auth methods and delete it. 2017-12-18 09:24:09 -05:00
Alena Volkova 90aa6c627e tests: Replace api_auth usages with auth methods in zerver/webhooks. 2017-12-18 09:24:09 -05:00
Alena Volkova 26eb23afc7 tests: Replace api_auth usages with auth methods in zerver/tests. 2017-12-18 09:24:09 -05:00
Alena Volkova 4a303d1778 tests: Add auth methods that will replace passing credentials. 2017-12-18 09:24:09 -05:00
Sivagiri 3792bd2d69 integrations: Add webhook payloads for Groove. 2017-12-14 18:54:22 -05:00
Andy Perez f0190333b8 bugdown: Remove whitespace strip from inline code.
Fixes #4507
2017-12-14 07:48:10 -05:00
Umair Khan 07961611b2 test_create_update_and_remove_default_stream_group: Fix flake. 2017-12-14 05:35:54 -05:00
Umair Khan b98c5c6d45 test_multi_user_subscription: Fix flake. 2017-12-14 05:35:54 -05:00
Vishnu Ks bc37317914 invites: Differentiate users invited as admin by showing bolt icon. 2017-12-13 15:12:21 -06:00
Greg Price 17a6632c43 zerver/views: Revert to Python 2 typing syntax for now (storage, streams).
This reverts commit 620b2cd6e.

Contributors setting up a new development environment were getting
errors like this:
```
++ dirname tools/do-destroy-rebuild-database
[...]
+ ./manage.py purge_queue --all
Traceback (most recent call last):
[...]
  File "/home/zulipdev/zulip/zproject/legacy_urls.py", line 3, in <module>
    import zerver.views.streams
  File "/home/zulipdev/zulip/zerver/views/streams.py", line 187, in <module>
    method_kwarg_pairs: List[FuncKwargPair]) -> HttpResponse:
  File "/usr/lib/python3.5/typing.py", line 1025, in __getitem__
    tvars = _type_vars(params)
[...]
  File "/usr/lib/python3.5/typing.py", line 277, in _get_type_vars
    for t in types:
TypeError: 'ellipsis' object is not iterable
```

The issue appears to be that we're using the `typing` module from the
3.5 stdlib, rather than the `typing=3.6.2` in our requirements files,
and that doesn't understand the `Callable[..., HttpResponse]` that
appears in the definition of `FuncKwargPair`.

Revert for now to get provision working again; at least one person
reports that reverting this sufficed.  We'll need to do more testing
before putting this change back in.
2017-12-13 10:20:11 -08:00
rht 620b2cd6e2 zerver/views: Use Python 3 syntax for typing (storage, streams). 2017-12-12 17:40:50 -08:00
Greg Price b830b446f1 logging: Reduce `create_logger` to new `log_to_file`.
The name `create_logger` suggests something much bigger than what this
function actually does -- the logger doesn't any more or less exist
after the function is called than before.  Its one real function is to
send logs to a specific file.

So, pull out that logic to an appropriately-named function just for
it.  We already use `logging.getLogger` in a number of places to
simply get a logger by name, and the old `create_logger` callsites can
do the same.
2017-12-12 17:17:08 -08:00
Greg Price 1c810baaef logging: Stop calling basicConfig in `create_logger`.
From the docs:

> This function does nothing if the root logger already has handlers
> configured for it.

Which we do if we've started up Django and configured settings, and in
particular allowed Django to process `settings.LOGGING`.

So, cut it out -- all it can do is confuse people about how logging
works.

If we ever actually used the `log_format` parameter, this would be
doubly confused, because only the first call would have any effect.
2017-12-12 17:17:08 -08:00
Greg Price b6491fd52a logging: Eliminate log_level from create_logger.
Because calls to `create_logger` generally run after settings are
configured, these would override what we have in `settings.LOGGING` --
which in particular defeated any attempt to set log levels in
`test_settings.py`.  Move all of these settings to the same place in
`settings.py`, so they can be overridden in a uniform way.
2017-12-12 17:17:08 -08:00
Greg Price ebcf0b4876 logging: Stop having `create_logger` force loglevels to INFO.
This is already the loglevel we set on the root logger, so this has no
effect -- except in tests, where `test_settings.py` attempts to set
some of these same loggers to higher loglevels.  Because the
`create_logger` call generally runs after we've configured settings,
it clobbers that effect.

The code in `test_settings.py` that tries to suppress logs only works
because it also sets `propagate=False`, which has nothing to do with
loglevels but does cause logs at this logger (and descendants) to be
dropped completely unless we've configured handlers for this logger
(or one of its relevant descendants.)
2017-12-12 17:17:07 -08:00
Weronika Grzybowska b54f78968f markdown: Add automatic numbering of ordered lists.
Adds a markdown preprocessor that finds ordered lists where all items
use the same number and change them to be in normal increasing order,
starting with that number.

Fixes #5159.
2017-12-12 18:35:38 -06:00
Steve Howell 0975bebacc quick fix: Fix migrations to be linear.
I merged a "good" but stale build, and the migrations
were out of order.  This fixes them.
2017-12-12 11:31:07 -06:00
Rhea Parekh acebd3a5e9 migrations: Replace special chars in stream & user names with space.
Also handle potential duplicate stream names created.
Fixes #6534
2017-12-12 10:04:31 -06:00
Rhea Parekh d30fb5a341 UserProfile: Disallow markdown character in the user profile name. 2017-12-12 10:04:31 -06:00
Rhea Parekh dd1b14378b Streams: Disallow certain markdown characters in stream names. 2017-12-12 10:04:31 -06:00
Rhea Parekh 8ed7f73199 IRC mirror: No longer require IRC stream names to start with #. 2017-12-12 10:04:31 -06:00
Greg Price 8a432c5134 APNs: Update comment on versions of our APNs payload format. 2017-12-11 20:55:20 -08:00
kunall17 98943a8333 Include data of stream/private sender details in apns push notifications 2017-12-11 20:51:09 -08:00
Rishi Gupta b7779667fc models: Set UserProfile.MIN_NAME_LENGTH to two.
Chinese names are often two characters.
2017-12-11 20:35:12 -08:00
Xavier Cooney ebc7c72d99 mypy: Use Python 3 type syntax in wordpress/view.py 2017-12-11 20:33:00 -08:00
Xavier Cooney 2db270be4b mypy: Use Python 3 type syntax in crashlytics/view.py 2017-12-11 20:33:00 -08:00
Xavier Cooney 1fa5233923 mypy: Use Python 3 type syntax in gitlab/tests.py 2017-12-11 20:33:00 -08:00
Xavier Cooney cc427911fa mypy: Use Python 3 type syntax in stripe/view.py 2017-12-11 20:33:00 -08:00
Xavier Cooney 7887857c81 mypy: Use Python 3 type syntax in user_settings.py 2017-12-11 20:33:00 -08:00
picapi_ 85ae723c9f mypy: Use Python 3 type syntax in zerver/lib/soft_deactivation.py. 2017-12-11 20:30:19 -08:00
picapi_ 2adeaab6ab mypy: Use Python 3 type syntax in zerver/views/invite.py. 2017-12-11 20:30:19 -08:00
picapi_ f07d39353a mypy: Use Python 3 type syntax in zerver/webhooks/basecamp/view.py. 2017-12-11 20:27:55 -08:00
picapi_ d038e7ea6b mypy: Use Python 3 type syntax in zerver/webhooks/bitbucket2/view.py. 2017-12-11 20:27:55 -08:00
picapi_ 20c2e53de4 mypy: Use Python 3 type syntax in zerver/webhooks/jira/view.py. 2017-12-11 20:27:55 -08:00
picapi_ 15fb392dd1 mypy: Use Python 3 type syntax in zerver/webhooks/papertrail/view.py. 2017-12-11 20:27:55 -08:00
picapi_ 3fb9803398 mypy: Use Python 3 type syntax in zerver/webhooks/teamcity/view.py. 2017-12-11 20:27:55 -08:00
Greg Price e9a7173119 actions: Cut some redundant types in docstrings.
We have a better way of writing types now!

Also move a comment that should have been a docstring,
to being a docstring.
2017-12-11 19:29:11 -08:00
Shreyansh Dwivedi 47fcb27e39 invitations: Remove custom_body.
Fixes #7672
2017-12-11 19:23:54 -08:00
Callum Fraser 00178d114b mypy: Use Python 3 type syntax in zerver/views/streams.py.
[greg: small formatting tweaks]
2017-12-11 18:43:24 -08:00
Callum Fraser aa9567ce37 mypy: Use Python 3 type syntax in zerver/middleware.py. 2017-12-11 18:43:24 -08:00
Rishi Gupta d9ce33dcf0 registration: Clarify realm_creation conditionals in accounts_register.
No change in behavior.
2017-12-11 18:36:14 -08:00
Rishi Gupta e545a681fc registration: Use standard pattern for returning link_does_not_exist.
Eventually this check for the realm will be done in get_object_from_key
itself. Rewriting this to fit the pattern in get_object_from_key.

No change to behavior.
2017-12-11 18:36:14 -08:00
Rishi Gupta dbe3576706 registration: Enforce realm is None only if realm_creation.
Commit d4ee3023 and its parent have the history behind this code.

Since d4ee3023^, all new PreregistrationUser objects, except those for
realm creation, have a non-None `realm`.  Since d4ee3023, any legacy
PreregistrationUsers, with a `realm` of None despite not being for
realm creation, are treated as expired.  Now, we ignore them
completely, and remove any that exist from the database.

The user-visible effect is to change the error message for
registration (or invitation) links created before d4ee3023^ to be
"link does not exist", rather than "link expired".

This change will at most affect users upgrading straight from 1.7 or
earlier to 1.8 (rather than from 1.7.1), but I think that's not much
of a concern (such installations are probably long-running
installations, without many live registration or invitation links).

[greg: tweaked commit message]
2017-12-11 18:36:14 -08:00
Ivche1337 1d202e30d8 integrations: Add webhook payloads for Opbeat. 2017-12-11 15:40:49 -06:00
Rishi Gupta 4eb3202b13 integrations: Remove weather example from zapier doc.
I don't think it made the page more readable, and doubled the length.
2017-12-11 15:39:26 -06:00
Arseny Chekanov e5c05f128a mypy: Use Python 3 type syntax in zerver/views/realm_filters.py. 2017-12-09 16:50:12 -08:00
Arseny Chekanov c2c023f0cf mypy: Use Python 3 type syntax in zerver/tornado/websocket_client.py. 2017-12-09 16:50:12 -08:00
Arseny Chekanov 1f34300006 mypy: Use Python 3 type syntax in zerver/webhooks/trello/tests.py. 2017-12-09 16:50:12 -08:00
Steve Howell 738c73f08e lint: Prevent `self: Any` annotations.
We should omit these for mypy.  For most class definitions,
mypy doesn't need `Any`, and it provides no real useful info.

For clever monkeypatches, you should provide a more specific
type than `Any`.
2017-12-08 19:00:18 -08:00
Steve Howell ac48772907 mypy: Improve typing for ManifestStaticFilesStorage monkeypatch. 2017-12-08 19:00:18 -08:00
Steve Howell f83a73c763 mypy: Use more specific type for _replacement_destroy_test_db. 2017-12-08 19:00:18 -08:00
Steve Howell 78041a5846 mypy: Use more specific type for instrument_url wrapper. 2017-12-08 19:00:18 -08:00
Xavier Cooney aa37b50d1e mypy: User Python 3 syntax for typing in realm.py. 2017-12-08 18:34:26 -08:00
Xavier Cooney a6465e8e35 mypy: Use Python 3 syntax in zapier/view.py. 2017-12-08 18:34:26 -08:00
Xavier Cooney 1262d14d39 mypy: Use Python 3 syntax in gci/view.py. 2017-12-08 18:34:26 -08:00
Xavier Cooney 38e65570ae mypy: Use Python 3 syntax for typing in heroku/view.py. 2017-12-08 18:34:26 -08:00
Xavier Cooney a831acd97f mypy: Use Python 3 type syntax in bugdown/__init__.py. 2017-12-08 18:34:26 -08:00
Xavier Cooney 63e793a8d1 mypy: Use Python 3 syntax for typing in views/tutorial.py. 2017-12-08 18:34:26 -08:00
Vishnu Ks a30152cb94 models: Use is_cross_realm_bot_email whenever possible.
Fixes #7557
2017-12-07 14:55:06 -08:00
Vishnu Ks ff43443ac7 models: Create is_cross_realm_bot_email function. 2017-12-07 14:55:06 -08:00
Rohitt Vashishtha 9c2a28b6df invitations: Fix lint errors in test_queue_workers. 2017-12-06 22:15:25 -08:00
Rishi Gupta 6787fbd547 invitations: Change rate limiting to use invites_sent::day.
[greg: small tweaks]
2017-12-06 20:35:50 -08:00
Rishi Gupta fbd8dde1f8 invitations: Add LoggingCountStat to keep track of sent invitations. 2017-12-06 20:35:50 -08:00
Rishi Gupta 100167fcf8 invitations: Change error messages for resend/revoke views.
We don't have a concept of invitation ID on the frontend (or backend).
2017-12-06 20:35:50 -08:00
Rishi Gupta cb5eb47470 invitations: Move resend/revoke error handling out of actions.py. 2017-12-06 20:35:50 -08:00
Rishi Gupta 968aae167b invitations: Remove get_prereg_user_by_email.
The original logic is buggy now that emails can belong to (and be
invited to) multiple realms.

The new logic in the `invites` queue worker also avoids the bug where
when the PreregistrationUser was gone by the time the queue worker got
to the invite (e.g., because it'd been revoked), we threw an exception.

[greg: fix upgrade-compatibility logic; add test; explain
revoked-invite race above]
2017-12-06 20:35:50 -08:00
Greg Price cab5c418f1 invitations: Add tests for queue worker. 2017-12-06 20:30:42 -08:00
Rishi Gupta 8baadc6b15 invitations: Add check for invite limit to resend invite pathway. 2017-12-06 20:30:42 -08:00
Greg Price b3482429f1 invitations: Adjust a few blank lines for clarity and compactness. 2017-12-06 20:30:42 -08:00
Rishi Gupta cb527fd3c6 invitations: Move spammer check from do_invite_users into its own function. 2017-12-06 20:30:42 -08:00
Rishi Gupta e48bc9136a invites: Change resend invitation pathway to use existing code.
This code changes frequently enough that errors are bound to creep in.  The
main change is that this sends the original invitation email instead of the
reminder email, but I think that's fine.
2017-12-06 20:30:42 -08:00
picapi_ 5febd177cf mypy: Use Python 3 type syntax in zerver/webhooks/newrelic/view.py. 2017-12-05 16:32:18 -08:00
picapi_ d55d680758 mypy: Use Python 3 type syntax in zerver/webhooks/homeassistant/view.py. 2017-12-05 16:32:18 -08:00
picapi_ c2ef7e7ce5 mypy: Use Python 3 type syntax in zerver/webhooks/freshdesk/view.py. 2017-12-05 16:32:18 -08:00
picapi_ 21e629f351 mypy: Use Python 3 type syntax in zerver/lib/test_classes.py. 2017-12-05 16:32:18 -08:00
picapi_ 67773a4b84 mypy: Use Python 3 type syntax in zerver/webhooks/semaphore/view.py. 2017-12-03 13:48:18 -08:00
picapi_ e2c14724d9 mypy: Use Python 3 type syntax in zerver/views/messages.py. 2017-12-03 13:48:18 -08:00
Rhea Parekh 460e9b4634 Scripts: Add script for resetting postgres sequences for imports. 2017-12-01 06:38:08 -08:00
Alena Volkova 3be8e95d13 settings: Send bot types from the backend instead of hardcoding them. 2017-11-30 15:28:53 -08:00
Tim Abbott 2310794451 bots: Don't allow embedded bots bot type in production.
This is important for making it easy for us to merge code related to
UI for embedded bots now, without enabling them in the production UI.
2017-11-30 15:28:53 -08:00
Rishi Gupta 7d1c88f0fb invitations: Limit realms to 100 invites per day.
To guard against using zulip invites as a vector for spam. Stopgap measure
until we figure out something better.
2017-11-29 22:18:05 -08:00
Rishi Gupta 7c9694077e confirmation: Move check_prereg_key_and_redirect to registration.py. 2017-11-29 22:18:05 -08:00
Tim Abbott 6e8f4ffc77 test_signup: Expand tests for invalid confirmation links.
We'll need the expanded test coverage when we move
check_prereg_key_and_redirect to zerver/views/registration.py to avoid
test failures, and these are also tests we should really have anyway.
2017-11-29 22:18:05 -08:00
Rishi Gupta 331a9bee6b confirmation: Add Confirmation.REALM_CREATION.
Does not change any behavior. Groundwork for adding a realm field to
Confirmation.
2017-11-29 22:03:09 -08:00
Greg Price 4926228071 rabbitmq: Do a better job of retrying failed connections.
Empirically, the retry in `_on_connection_closed` didn't actually work
-- if a reconnect failed, that was it, and the exception handler
didn't get run.  A traceback would get logged, but all its frames were
in Tornado or Pika, not our own code; presumably something magic and
async was happening to the exception.

Moreover, though we would make one attempt to reconnect if we had a
connection that got closed, we didn't have any form of retry if the
original attempt at connecting failed in the first place.

Happily, upstream offers a perfectly reasonable bit of API that avoids
both of these problems: the on-open-error callback.  So use that.
2017-11-29 16:56:29 -08:00
Greg Price 7ac2b58584 rabbitmq: Reorder a bit to group our reconnect logic together. 2017-11-29 16:56:29 -08:00
Greg Price c32b16715d tornado: Use spiffy new `call_later` rather than `add_timeout`.
This method was new in Tornado 4.0.  It saves us from having to get
the time ourselves and do the arithmetic -- which not only makes the
code a bit shorter, but also easier to get right.  Tornado docs (see
http://www.tornadoweb.org/en/stable/ioloop.html) say we should have
been getting the time from `ioloop.time()` rather than hardcoding
`time.time()`, because the loop could e.g. be running on the
`time.monotonic()` clock.
2017-11-29 16:56:29 -08:00
Greg Price 73886f57d2 zerver/lib/queue: Clean up import order. 2017-11-29 16:56:29 -08:00
Greg Price 3c4e4c14c9 rabbitmq: Add on-close callback atomically in creating the connection.
Adding it afterward is inherently racy, and upstream's API is quite
reasonable for avoiding that -- just like we can pass an on-open
callback up front, we can do the same with the on-close callback.

This is a more thorough version of 4adf2d5c2 from back in 2013-04.
2017-11-29 16:56:29 -08:00
Greg Price e88c2a7ee4 rabbitmq: Cut redundant `stop_ioloop_on_close` parameter.
The default value of this parameter is already False upstream.
(It was already False in pika version 0.9.6, which we were
supposedly using when we introduced this in 4baeaaa52; not sure
what the story was there.)
2017-11-29 16:56:29 -08:00
Tim Abbott be832387c5 user_groups: Fix reporting of empty user groups.
Previously, we weren't doing a proper left join in
user_groups_in_realm_serialized, resulting in empty user groups being
excluded from the query.  We want to leave decisions about excluding
empty user groups to the UI layer, so we include these here.
2017-11-29 16:13:55 -08:00
Greg Price fe979af822 purge_queue: Allow purging Tornado-consumed queues.
These are the exceptions to the rule that our queues correspond to
queue-processor workers.

Purging `notify_tornado` in particular is a useful workaround right
now for some error spew in the dev environment.
2017-11-29 14:44:53 -08:00
Tim Abbott b2cb443d24 subs: Fix clearing unread counts when leaving private streams.
Because we use access_stream_by_id here, and that checks for an active
subscription to interact with a private stream, this didn't work.

The correct fix to add an option to active_stream_by_id to accept an
argument indicating whether we need an active subscription; for this
use case, we definitely do not.
2017-11-29 14:40:08 -08:00
Tim Abbott 6bc1a682de notifications: Separate push and email notifications checks.
This is an early step in a larger refactor to properly decouple the
email and push notification code paths.
2017-11-28 17:51:18 -08:00
Sarah ba70b542d3 notification settings: Add stream email notifications setting.
Adds a field to UserProfile for enabling emails about stream
messages and a field to Subscriptions about email
notification preferences.
2017-11-28 17:51:18 -08:00
rht ee546a33a3 zerver/lib: Use python 3 syntax for typing.
Edited by tabbott to improve various line-wrapping decisions.
2017-11-28 17:15:14 -08:00
rht 229a8b38c0 zerver/lib: Use Python 3 syntax for typing for several files.
This adds a number of annotations that had been missed in previous
passes.
2017-11-28 17:02:24 -08:00
rht 115f7e6055 zerver/tornado: Use Python 3 syntax for typing (part 2). 2017-11-28 17:02:24 -08:00
rht a1cc720860 zerver: Use Python 3 syntax for typing.
Tweaked by tabbott to fix some minor whitespace errors.
2017-11-28 16:49:36 -08:00
Greg Price 0ec2a9d259 auth: Try switching to register even if user exists on another realm.
For example, this means that if a user already has an account on one
realm and they try to make an account on another by hitting "Sign in
with Google" (rather than following the little "Register" link to a
"Sign up with Google" button instead), they'll get to make an account
instead of getting an error.

Until very recently, if the user existed on another realm, any attempt
to register with that email address had to fail in the end, so this
logic gave the user a useful error message early.  We introduced it in
c23aaa178 "GitHub: Show error on login page for wrong subdomain"
back in 2016-10 for that purpose.  No longer!  We now support reusing
an email on multiple realms, so we let the user proceed instead.

This function's interface is kind of confusing, but I believe when its
callers use it properly, `invalid_subdomain` should only ever be true
when `user_profile` is None -- in which case the revised
`invalid_subdomain` condition in this commit can never actually fire,
and the `invalid_subdomain` parameter no longer has any effect.  (At
least some unit tests call this function improperly in that respect.)
I've kept this commit to a minimal change, but it would be a good
followup to go through the call sites, verify that, eliminate the use
of `invalid_subdomain`, then remove it from the function entirely.
2017-11-28 16:38:41 -08:00
Vishnu Ks d329636079 tests: Add test for user signup with already-used email.
[Test simplified a bit by greg.]
2017-11-28 16:23:10 -08:00
Greg Price 50e451cd7b populate_db: Create an email-reusing Cordelia in the "lear" realm. 2017-11-28 16:23:10 -08:00
Tim Abbott 586e18b237 auth: Allow accounts with the same email in different realms.
[Modified by greg to (1) keep `USERNAME_FIELD = 'email'`,
(2) silence the corresponding system check, and (3) ban
reusing a system bot's email address, just like we do in
realm creation.]
2017-11-28 16:23:10 -08:00
Tim Abbott e6f460f511 auth: Replace user_email_is_unique validator.
As we migrate to allow reuse of the same email with multiple realms,
we need to replace the old "no email reuse" validators.  Because
stealing the email for a system bot would be problematic, we still ban
doing so.

This commit only affects the realm creation logic, not registering an
account in an existing realm.
2017-11-28 16:23:10 -08:00
Greg Price d6cfa56bc1 test_signup: Test that cloning a system bot's email is not allowed.
Just now this is largely redundant with `test_signup_already_active`;
but very soon when we allow reusing an email across realms, the logic
will diverge.
2017-11-28 16:23:10 -08:00
Greg Price 5f38b374b6 register: Activate existing mirror dummy only if on same realm.
The one thing this bit of logic is used for is to decide whether
there's an existing user which is a mirror dummy that we should
activate.  This change causes us to ignore such an existing user if
it's on some other realm, and go straight into `do_create_user`.
2017-11-28 16:23:10 -08:00
Greg Price eb8b05b2b5 test_email_mirror: Remove a brittle bit of logic.
This code appears to exist to cover a few extra lines in
zerver/lib/digest.py.  But it's rather brittle, tucked as it is into
the middle of a different test's loop, and with the upcoming
introduction of the `lear` realm in testing, this test code itself
loses coverage.

For now, rather than fix this test code up just delete it; we don't
have 100% coverage on `zerver/lib/digest.py`, while we do on this test
file, so that avoids breaking coverage in CI.  As a followup, we
should add back some logic like this but in a more robust way,
probably as its own separate test method.
2017-11-28 16:23:10 -08:00
Greg Price 453585abb1 test_messages: Clean up some error spew by updating a test.
This test would produce a bunch of log messages with tracebacks,
complaining that `welcome-bot@zulip.com` tried to send cross-realm PMs
and can't.  The issue is that the test overrides
`settings.CROSS_REALM_BOT_EMAILS`, and hasn't kept up with additions
to the normal value for that setting.  Update it so that welcome-bot
is permitted as usual.
2017-11-28 16:23:10 -08:00
Greg Price b6cc21b438 debug: Add facility to dump tracemalloc snapshots.
Originally this used signals, namely SIGRTMIN.  But in prod, the
signal handler never fired; I debugged fruitlessly for a while, and
suspect uwsgi was foiling it in a mysterious way (which is kind of
the only way uwsgi does anything.)

So, we listen on a socket.  Bit more code, and a bit trickier to
invoke, but it works.

This was developed for the investigation of memory-bloating on
chat.zulip.org that led to a331b4f64 "Optimize query_all_subs_by_stream()".

For usage instructions, see docstring.
2017-11-28 15:52:07 -08:00
derAnfaenger 6dd639454e integrations: Add topic parameter for Appfollow.
Tweaked by tabbott to document the integration.
2017-11-28 15:49:19 -08:00
Umair Khan 1acdfef13c two_factor: Disable prefix in OurAuthenticationForm.
In two factor authentication every step adds a unique prefix to the fields,
due to this the name of the form fields differs from the HTML fields. If
we do not do this we will have to change the name in the HTML, which
will cause the change in tests.
2017-11-28 15:27:44 -08:00
Tim Abbott 4f5a5a8547 auth: Extract add_dev_login_context.
This simple refactor cleans up the code for the dev_auth_enabled code
path to be a bit less deeply in the login_page() logic.
2017-11-28 15:17:33 -08:00
Tim Abbott d5bb45db60 webhooks/gci: Fix missing mypy annotation. 2017-11-28 14:08:44 -08:00
Sampriti Panda bea653fabc webhooks/gci: Improve message formatting.
* Use Student Name instead of Task Name in subject.
* Use Task Instance URL instead of Task Definition URL (and workaround
  for a bug in the API).
2017-11-28 13:53:42 -08:00
Robert Hönig 1e66bd1c02 tests: Add newlines to test_service_bot_system.test_internal_endpoint(). 2017-11-27 21:05:34 -08:00
derAnfaenger c8a5ae753c embedded bots: Consistently use 'storage' instead of 'state.' 2017-11-27 21:05:34 -08:00
Tim Abbott 9645c8b31f integrations: Remove disabled googlesearch integration.
This was moved to another part of the python-zulip-api codebase
because it was unfinished.
2017-11-27 20:50:37 -08:00
Eeshan Garg c45517f544 python-zulip-api: Upgrade to PyPI package release 0.3.8.
There's one migration required by this release:

* queue_processors: Stop passing state_handler to handle_message.

  state_handler is now a property of bot_handler and thus, does
  not need to be passed to bot_handler.handle_message().

  The commit responsible is:
  2a74ad11c5
2017-11-27 20:31:37 -08:00
rht e538f4dd44 zerver/views: Use Python 3 syntax for typing.
Edited by tabbott to remove state.py and streams.py, because of
problems with the original PR's changes, and wrap some long lines.
2017-11-27 17:10:39 -08:00
Tim Abbott bd8196c3a5 lint: Fix too-long line in auth.py. 2017-11-27 15:28:46 -08:00
Vishnu Ks d4ee3023b2 registration: Require an explicit realm on PreregistrationUser.
This completes the last commit's work to fix CVE-2017-0910, applying
to any invite links already created before the fix was deployed.  With
this change, all new-user registrations must match an explicit realm
in the PreregistrationUser row, except when creating a new realm.

[greg: rewrote commit message]
2017-11-27 14:59:34 -08:00
Vishnu Ks 985768b2fd registration: Check realm against PreregistrationUser realm.
We would allow a user with a valid invitation for one realm to use it
on a different realm instead.  On a server with multiple realms, an
authorized user of one realm could use this (by sending invites to
other email addresses they control) to create accounts on other
realms. (CVE-2017-0910)

With this commit, when sending an invitation, we record the inviting
user's realm on the PreregistrationUser row; and when registering a
user, we check that the PregistrationUser realm matches the realm the
user is trying to register on.  This resolves CVE-2017-0910 for
newly-sent invitations; the next commit completes the fix.

[greg: rewrote commit message]
2017-11-27 14:58:26 -08:00
Tim Abbott 8b935f4e99 settings: Add setting for SYSTEM_BOT_REALM.
This fixes some subtle JavaScript exceptions we've been getting in
zulipchat.com, caused by the system bot realm there not being "zulip"
interacting with get_cross_realm_users.
2017-11-27 14:46:07 -08:00
Tim Abbott 25fd4c5508 bulk_get_users: Edit the cache keys to make them more unique.
While at this point I was to rewrite this function, this at least
plugs the issues for now.
2017-11-27 14:41:31 -08:00
Tim Abbott 339b67f7af get_cross_realm_dicts: Harden against caching bug in last commit.
This should help protect us from future issues with the way that
`bulk_get_users` does caching.

It's likely that we'll want to further restructure `bulk_get_users` to
not have this base_query code path altogether (since it's kinda
buggy), but I'm going to defer that for a time when we have another
user.
2017-11-27 14:35:50 -08:00
Tim Abbott 646ba5b9e5 bulk_get_users: Fix issues with users in multiple realms.
The previous implementation had a subtle caching bug: because it was
sharing its cache with the `get_user_profile_by_email` cache, if a
user happened to have an email in that cache, we'd return it, even
though that user didn't match `base_query`.

This causes `get_cross_realm_users` to no longer have a problematic
caching bug.
2017-11-27 14:34:45 -08:00
Rohitt Vashishtha 6734b00c0b markdown: Hide URL if message is only an image.
Hides URL if the message content == image url so that sending gifs or
images feels less cluttered. Uses the url_to_a() function to generate
the expected url string for matching.

Fixes #7324.
2017-11-27 13:30:18 -08:00
Rohitt Vashishtha 3550467899 markdown: Insert text in link-only inline image preview tests.
Appends "Test: " text to some tests to make changes to the image preview
rendering. In the future, if the message is only a link to an image,
the link will be hidden.
2017-11-27 13:30:18 -08:00
Tim Abbott 649b7b8b9f internal_send_message: Enforce being passed cross-realm bots.
We include ERROR_BOT in this set, even though it's not technically
cross-realm (it just lives in the admin realm).

This code path does not correctly handle emails that correspond to
multiple accounts (because `get_system_bot` does not).  Since it's
intended to only be used by system bots, we add an appropriate
assertion to ensure it is only used for system bots.
2017-11-26 17:14:23 -08:00
Tim Abbott ae8e203717 actions: Merge internal_prep_message into internal_send_message.
We also document the fact that only system bots are supported here.
2017-11-26 17:14:23 -08:00
Tim Abbott 6952dcbdac embedded bots: Stop using internal_send_message for non-system-bots.
This was causing problems, because internal_send_message assumes that
there is a unique user (across all realms) with the given email
address (which is sorta required to support cross-realm bot messages
the way it does).

With this change, it now, in practice, only sends cross-realm bot
messages.
2017-11-26 17:14:23 -08:00
Tim Abbott 27582782fa embedded bots: Flip around ordering of rate-limiting check. 2017-11-26 17:14:23 -08:00
Tim Abbott 5306a9634d email_mirror: Rewrite to not use internal_send_message.
This was causing problems with the fact that `get_system_bot` now only
works for actual system bot users.
2017-11-26 17:14:23 -08:00
Eeshan Garg 5b8f38f4e3 webhooks/bitbucket2: Ignore push events with no changes.
We now ignore payloads where payload['push']['changes'] is empty,
because an empty push doesn't really convey any useful information.

I couldn't find a way to replicate the action that would generate
such a payload, so I took one of our existing payloads and editted
out payload['push']['changes'] myself, so this payload is not
authentic.
2017-11-26 17:03:07 -08:00
Tim Abbott 2210f627a5 signup: Switch active mirror-dummy users to an AssertionError.
Previously, this was a ValidationError, but that doesn't really make
sense, since this condition reflects an actual bug in the code.

Because this happened to be our only test coverage the ValidationError
catch on line 84 of registration.py, we add nocoverage there for now.
2017-11-26 16:13:45 -08:00
Vishnu Ks b9bc1c2b33 Eliminate get_user_profile_by_email from test_classes. 2017-11-26 15:47:56 -08:00
Vishnu Ks 610eb557b8 backend: Make password reset form support multi realm membership. 2017-11-26 15:35:25 -08:00
Vishnu Ks 9f225360dc register: Make /register redirect to find accounts in root domain without realm.
If there is no realm in the root domain don't show
registration form. Instead just redirect to find
accounts page.
2017-11-26 15:32:17 -08:00
Vishnu Ks bab0b913ee test_signup: Extract logic for realm-creation test to a function. 2017-11-26 15:29:16 -08:00
Vishnu Ks 1b5753d021 test_signup: Fix assertIsNone check for realm in RealmCreationTest. 2017-11-26 15:27:33 -08:00
Tim Abbott 9fcdb6c83a notifications: Use call_consume_in_tests for email sending.
This adds a bit of test coverage with really no downside.
2017-11-26 11:46:07 -08:00
Robert Hönig 0e0a8a2b14 queue processor tests: Call consume by default.
This significantly improves the API for queue_json_publish to not be
overly focused on what the behavior of this function should be in our
unit tests.
2017-11-26 11:45:34 -08:00
Eeshan Garg 23162a1433 github_webhook: Use indented Markdown macros. 2017-11-25 17:01:11 -08:00
Eeshan Garg 1dca7cc367 webhooks/beanstalk: Use indented Markdown macros. 2017-11-25 17:01:11 -08:00
Tim Abbott 6968b540c8 mypy: Remove type: ignores that are unnecessary with new mypy.
Mostly these can be removed because the broken LXML stubs were removed
from typeshed in https://github.com/python/typeshed/issues/525.
2017-11-25 10:06:28 -08:00
Rhea Parekh 84847b811b requirements: Upgrade mypy to 0.550.
We have to add a few type: ignores due to mypy 0.550 having some bugs
in the new imaplib stubs in typeshed.

Fixes #7387.
2017-11-25 10:06:27 -08:00
Vaibhav Sagar 711f7bca3b upload: Fix typo in animated GIF error message.
s/must be have/must have the/.
2017-11-24 09:08:38 -08:00
Andy Perez e7f1825617 bugdown: Update KaTeX to version 0.8.3 from 0.7.1.
Fixes #7480.
2017-11-23 12:43:34 -08:00
Tim Abbott 938892db8f lint: Ban tab-based whitespace in our our text fixtures.
We don't want to do this for the webhook fixtures since those come
from third-party code, but it makes sense to do this for hand-edited
code.
2017-11-23 10:47:09 -08:00
Andy Perez 0449dc8b8a bugdown: Fix handling of nested fenced math blocks.
Change any fence delimiters (backticks and tildes) to be
interpreted literally inside of math fenced_code blocks.

Fixes #5190.
2017-11-22 12:19:43 -08:00
Tommy Ip e44b8809b2 /team: Render contributors list in tabs client side. 2017-11-22 11:33:24 -08:00
Tommy Ip 019c06a087 /team: Populate authors.json with updated file structure. 2017-11-22 11:33:24 -08:00
derAnfaenger 43484ad494 markdown processor: Exclude some files from macro substitution.
This fixes unintended macro substitution for files explaining
the macro system.
2017-11-22 10:29:07 -08:00
Harshit Bansal 2e7d0cb4cb digests: Fix new stream links.
Fixes: #7479.
2017-11-22 10:07:36 -08:00
Tim Abbott 8f6b39a1c0 mypy: Fix annotations for json_encoder_for_html. 2017-11-22 01:32:46 -08:00
rht 3ec90f8b33 zerver/tests: Use python 3 syntax for typing (final). 2017-11-21 22:01:19 -08:00
rht 0260ba82ca zerver/tests: Use python 3 syntax for typing (part 4). 2017-11-21 22:01:19 -08:00
rht 3bf9cd0656 zerver/tests: Use python 3 syntax for typing (part 3). 2017-11-21 22:01:19 -08:00
rht e3daa09b05 zerver/tests: Use python 3 syntax for typing. 2017-11-21 22:01:19 -08:00
rht e169bb0954 zerver: Remove simplejson dependency.
Modified by tabbott to put the third-party code in a new file.

Fixes #6970.
2017-11-21 21:51:48 -08:00
Tim Abbott b4e67fac36 bot_lib: Fix storage bug introduced by rebase error. 2017-11-21 21:37:54 -08:00
rht 5f6dbe6226 fixtures: replace optparse example with argparse. 2017-11-21 21:23:41 -08:00
derAnfaenger e526d0c144 embedded bots: Add views to access state. 2017-11-21 21:10:39 -08:00
derAnfaenger d2af8d4cbd embedded bots: Support multi-entry state modification. 2017-11-21 21:10:39 -08:00
derAnfaenger cbadd3d430 embedded bots: Simplify StateError exception messages. 2017-11-21 21:10:12 -08:00
Tim Abbott 141cf21b86 bulk_create: Fix buggy logic for generating recipients_by_email.
This buggy logic from e1686f427c had
broken do-destroy-rebuild-test-database.

Now that we're not just trying to add the Recipient objects for every
user on the system here to profiles_by_id, we also shouldn't be
processing every Recipeint object on the system.  The fix is simple:
because of the patch we got merged into Django upstream,
recipients_to_create actually has the object IDs added to the
Recipient objects passed into Recipient.objects.bulk_create.

This was missed in manual testing, since it only broke `populate_db
--test-suite`.
2017-11-21 21:07:32 -08:00
rht 3f4bf2d22f zerver/lib: Use python 3 syntax for typing.
Extracted from a larger commit by tabbott because these changes will
not create significant merge conflicts.
2017-11-21 20:56:40 -08:00
rht 561ba33f69 zerver/lib: Use python 3 syntax for typing.
Split by tabbott from a larger commit; this covers a batch of files
with no open PRs touching them.
2017-11-21 20:45:52 -08:00
Eeshan Garg 73d0f1db81 lib/integrations: Remove Integration.doc_context.
An Integration object doesn't need access to the context dict used
to render its doc.md, since the context dict is just passed directly to
render_markdown_path.
2017-11-21 20:39:30 -08:00
Eeshan Garg 600ee90a38 integrations: Add add_context_for_single_integration function.
Previously, when rendering a single integration, we tacked on the
following information to the context dict that was redundant:

* An OrderedDict containing all of the Integration objects for
  all integrations.
* An OrderedDict containing all of the integration categories.

The context dict for rendering a particular integration doc would
contain 4 OrderedDicts, 2 for categories, 2 for Integration objects
because of how many times add_integrations_context had been called.

This was very wasteful, since an Integration object doesn't need
to access any other Integration object (or itself for that matter)
to render its documentation. This commit adds a function that
allows us to only pass in the context values that are necessary.
2017-11-21 20:39:30 -08:00
Tim Abbott e1686f427c bulk_create: Remove assumption that UserProfiles are globally unique.
This isn't used in production, but that could change in the future,
and the fix to make this limited to a single realm is pretty simple.
2017-11-21 20:23:06 -08:00
Tim Abbott f6e57fd514 forms: Remove unnecessary OurAuthenticationForm logic.
This is checked for in the caller of OurAuthenticationForm, which
meant this code was never run.  But it is worth having an assertion
here to catch any possible regressions.
2017-11-21 20:14:12 -08:00
Tim Abbott 719d6c49df forms: Stop using get_user_profile_by_email in OurAuthenticationForm.
Structurally, the main change here is replacing the `clean_username`
function, which would get called when one accessed
self.cleaned_data['username'] with code in the main `clean` function.

This is important because only in `clean` do we have access to the
`realm` object.

Since I recently added full test coverage on this form, we know each
of the major cases have a test; the error messages are unchanged.
2017-11-21 20:14:12 -08:00
Tim Abbott 665fc594db auth: Set valid_attestation more unconditionally in social auth. 2017-11-21 20:14:12 -08:00
Tim Abbott ade5b4ea69 auth: Convert SocialAuthMixin to accept a realm object. 2017-11-21 20:14:12 -08:00
Tim Abbott 0377288f68 auth: Fix GitHub test for invalid user account. 2017-11-21 20:14:12 -08:00
Tim Abbott 251a537e0d auth: Set the subdomain in more GitHub auth tests.
This should have been set in basically all of these tests to set them
up properly, and this issue will matter after upcoming changes.
2017-11-21 20:14:12 -08:00
Tim Abbott 732dd1b6a3 auth: Improve logic for invalid GitHub emails.
This deletes the old mock-covered test for this, which was mostly
useless.  We have a much less messy test, which we extend to provide
the same test coverage the old one did.

While the result was the same before, this makes it more obvious.
2017-11-21 20:14:12 -08:00
Vishnu Ks 766511e519 actions: Mark all messages as read when user unsubscribes from stream.
This fixes a bug where, when a user is unsubscribed from a stream,
they might have unread messages on that stream leak.  While it might
seem to be a minor problem, it can cause significant problems for
computing the `unread_msgs` data structures, since it means we need to
add an extra filter for whether the user is still subscribed, either
in the backend or in the UI.

Fixes #7095.
2017-11-21 20:09:17 -08:00
Tim Abbott e0b56c72de ldap: Simplify logic for user creation.
self._realm can't be None here with the new logic in authenticate().
2017-11-21 19:08:45 -08:00
Tim Abbott c4c8879cf7 ldap: Fix the error message for deactivated users. 2017-11-21 18:35:05 -08:00
Tim Abbott 3bfb19b5f3 Convert EmailAuthBackend and LDAPAuthBackend to accept a realm. 2017-11-21 18:23:50 -08:00
Tim Abbott 1b95b098dd auth: Clarify comments explaining the GoogleMobileOauth2Backend. 2017-11-21 18:23:50 -08:00
Tim Abbott a7d51127fb auth: Convert GoogleMobileOAuth2Backend to accept a realm object. 2017-11-21 18:23:49 -08:00
Tim Abbott 07bc31f818 auth: Convert DevAuthBackend to accept a realm object. 2017-11-21 18:23:49 -08:00
Tim Abbott 4968631d1b auth: Convert DevAuthBackend to use a unique argument pattern.
This helps ensure that we won't accidentally activate this backend on
other code paths.
2017-11-21 18:23:49 -08:00
Tim Abbott d63e9f240c auth: Remove unnecessary remote_user=None code path.
This code path was only required because we had remote_user set as a
positional argument here, and thus we'd be running this auth backend's
code when actually using another auth backend (due to how Django auth
backends are selected based on argument signature).
2017-11-21 18:23:49 -08:00
Tim Abbott 387c9109ec auth: Convert RemoteUserBackend to accept a realm object. 2017-11-21 18:23:49 -08:00
Tim Abbott f17974ab32 DummyAuthBackend: Require being passed a realm object.
We should now always know the realm in our auth code paths.
2017-11-21 18:22:37 -08:00
Vishnu Ks 34689370cd settings: Create UI and backend for setting signups stream. 2017-11-21 17:39:51 -08:00
Vishnu Ks 3d2c9c6098 models: Replace core team with Realm.INITIAL_PRIVATE_STREAM_NAME. 2017-11-21 17:39:51 -08:00
Vishnu Ks a0275a6257 actions: Send new user messages to signup_notifications_stream. 2017-11-21 17:39:50 -08:00
Rishi Gupta 42652713c8 onboarding: Refactor away setup_initial_private_stream.
It is a confusing name, now that the stream creation part has been taken
away.
2017-11-21 17:39:50 -08:00
Vishnu Ks 3d17897563 onboarding: Move signup notifications stream creation to do_create_realm. 2017-11-21 17:39:50 -08:00
Vishnu Ks 08aca174fe models: Add signup_notifications_stream attribute to Realm. 2017-11-21 17:39:50 -08:00
Tim Abbott 29408d24ff emoji: Rework reactions validation to have a single function.
This feels more readable to me, and also identified a couple cases
where we were missing test coverage.
2017-11-20 15:27:21 -08:00
Tim Abbott 2d4c7b771a reactions: Move legacy endpoint code further down in file.
This helps ensure that new contributors working on the reactions code
read the stuff we use now first.
2017-11-20 15:27:21 -08:00
Tim Abbott 80a2a36dd1 reactions: Rename legacy reactions for greater clarity. 2017-11-20 15:27:21 -08:00
Harshit Bansal 87b523f3ef emoji.py: Refactor emoji data consistency checking functions.
Inorder to provide more explicit error messages I have merged the
`emoji_code_is_valid()` and `emoji_name_is_valid()` functions into
`check_emoji_code_consistency()` and `check_emoji_name_consistency()`
respectively.
2017-11-20 15:27:21 -08:00
Harshit Bansal 9a4d5fe1f8 test_reactions.py: Extract `EmojiBaseReaction` test class. 2017-11-20 15:27:20 -08:00
Rishi Gupta 27babcf92b portico: Update error message for deactivated user.
The installation admin is not the right person to get support requests from
deactivated users, regardless of the situation.

Also updates the wording to be a bit more concise.
2017-11-20 13:40:51 -08:00
Tim Abbott 25ddba99f6 lint: Ban general use of user_profile.save().
This often can cause minor caching problems.

Obviously, it'd be better if we had access to the AST and thus could
do this rule for UserProfile objects in general.
2017-11-20 10:57:08 -08:00
Vishnu Ks 432c9d3449 email_log: Reset the value of email after email change process.
This commit actually needs only user_profile.save() line
but it's more readable with both the lines.
2017-11-20 10:45:16 -08:00
Tim Abbott 0667a62244 password reset: Simplify password reset form logic.
Now that we're generating the URL inside the Python code, we can clean
up the context logic.
2017-11-20 10:40:33 -08:00
Vishnu Ks 9c50819dd6 email: Add reset button to password reset email. 2017-11-20 10:35:03 -08:00
Tim Abbott c8edbae21c password reset: Fix error message for invalid realm.
This is a lot cleaner than the previous model.

Basically rewritten by Vishnu Ks to actually work :).
2017-11-20 10:34:55 -08:00
Tim Abbott 10fbafa13e PasswordResetTest: Clean up mocking of logging.info. 2017-11-20 10:32:40 -08:00
Umair Khan 95ba3e7cbb password_reset: Send email unconditionally.
This was basically rewritten by tabbott, because the code is a lot
cleaner after just rewriting the ZulipPasswordResetForm code to no
longer copy the model of the original Django version.

Fixes #4733.
2017-11-20 10:32:40 -08:00
Eeshan Garg 02a32e246e webhooks/gitlab: Support test payloads without an "action" attribute.
Payloads that don't have a payload['object_attributes']['action']
attribute are generated when GitLab sends a test payload to verify
if the webhook was set up successfully. In this case, we should
send a message notifying that the webhook was configured
successfully.
2017-11-19 20:56:10 -08:00
Eeshan Garg 9138d3b8d7 integrations: Move integration-specific context to integration_doc.
Instead of populating the context dict with integration-specific
information in render_markdown_path, we now do that in
zerver.views.integrations.integration_doc instead.

Fixes #7401.

Tweaked by tabbott to use cast to handle the typing issues here.
2017-11-19 20:48:46 -08:00
Shubham Dhama 8b152fc109 email: Fix identification of chrome as safari on ios
The user agent(UA) string for chrome on ios contains "crios" instead of "chrome".
Added the missing test for corresponding case.
Fixes: #7441.
2017-11-19 17:07:33 -08:00
Rishi Gupta f6bf11f5e6 portico: Add for/mystery-hunt. 2017-11-19 17:04:17 -08:00
rht 33b1a541d7 zerver/lib: Use python 3 syntax for typing.
With tweaks by tabbott to fix line spacing.
2017-11-18 16:09:04 -08:00
rht 4a07214725 zerver/tests: Use python 3 syntax for typing. 2017-11-18 15:41:16 -08:00
rht 6c28c7e879 zerver/tests: Use python 3 syntax for typing. 2017-11-18 15:35:57 -08:00
Tim Abbott 18369cb5ad JWT: Fetch the subdomain earlier in the auth code path. 2017-11-17 17:33:28 -08:00
Tim Abbott 462b7a1b3c tests: Expand test coverage of OurAuthenticationForm.
This adds tests for a new more cases.  Some were already covered
elsewhere in the codebase, but it feels best for LoginTest to fully
cover OurAuthenticationForm.
2017-11-17 17:31:38 -08:00
Tim Abbott 81adc92db6 actions: Use get_system_bot for fetching WELCOME_BOT. 2017-11-17 17:25:56 -08:00
Tim Abbott cd95c09fca forms: Use an AssertionError for unexpected DNS errors. 2017-11-17 17:25:56 -08:00
Tim Abbott 1ed50ee858 tests: Pass a realm option to login for non-zulip realms.
This better matches the model of how having multiple realms should
work: you need to specify which realm you're logging into.
2017-11-17 15:32:42 -08:00
Tim Abbott 1e334f41ed test_subs: Fix nondeterministic failures in default stream tests.
Here we sort the streams before comparing them.
2017-11-17 15:32:42 -08:00
Greg Price bd5905da5d error_notify: Use `git describe --always`.
This way, even if the repo lacks any tags, we still
at least get the commit ID.
2017-11-17 14:19:20 -08:00
Andy Perez 83f563aeff bugdown: Properly parse empty blockquotes.
The character ">" now only starts a blockquote if the resulting
blockquote would be non-empty.  Thus, by itself, ">" is now
interpreted literally by bugdown, fixing #687.  The message
with contents consisting of ">>>" is now parsed as a doubly
(not triply) nested blockquote with contents ">".  Properly
formed blockquotes have identical behavior as before, but now
bugdown can no longer produce empty blockquotes as output.

Fixes #2886, #687.
2017-11-17 11:41:44 -08:00
derAnfaenger 6049cd27b1 migrations: Fix conflict. 2017-11-17 08:24:56 -08:00
derAnfaenger 395f1e9270 embedded bots: Add database config storage.
Storage limititations are only set on the value of
a config entry, since this is the only user-accessible
part of the schema. Keys are statically set by each
embedded bot.
2017-11-16 23:06:38 -08:00
Tim Abbott 8dc82f97c7 python: Wrap long def lines in test files.
We don't have our linter checking test files due to ultra-long strings
that are often present in test output that we verify.  But it's worth
at least cleaning out all the ultra-long def lines.
2017-11-16 22:00:53 -08:00
rht 4f5b1c0a5a zerver/tests: Use python 3 syntax for typing in most files. 2017-11-16 21:52:01 -08:00
Vishnu Ks 068912ca7d tests: Add test for default stream group signup flow. 2017-11-16 21:17:31 -08:00
Vishnu Ks cc553125a1 tests: Move check_user_subscribed_only_to_streams to test_classes ZulipTestCase. 2017-11-16 21:17:31 -08:00
Vishnu Ks 1b9b142d51 test_signup: Store return value of sorted.
Sorted does not sort in-place.
2017-11-16 21:17:31 -08:00
Rishi Gupta f0807353ae settings: Remove ABOUT_LINK_DISABLED.
Was introduced in 0f4bbc8, I think because the /team page was unfinished at
the time.
2017-11-16 21:15:24 -08:00
Harshit Bansal d9c2f613e3 api: Add new endpoint for reactions.
This endpoint will allow us to add/delete emoji reactions whose emoji
got renamed during various emoji infra changes. This was also a
required change for realm emoji migration.

This commit was tweaked significantly by tabbott for greater clarity
(with no changes to the actual logic).
2017-11-16 20:52:15 -08:00
Greg Price 3828305305 error_notify: Try `git describe` too.
Tested for graceful degradation both with `git` not existing and the
`.git` directory not existing.
2017-11-16 19:15:12 -08:00
Greg Price 41ecdd8d83 error_notify: Add some idea of the deployed code version.
Tested in dev with the help of our handy /emails/ display.
2017-11-16 19:15:12 -08:00
Greg Price 4b5c52fc99 queue: Prevent an AttributeError rather than swallowing it.
When the RabbitMQ server disappears, we log errors like these:

```
Traceback (most recent call last):
  File "./zerver/lib/queue.py", line 114, in json_publish
    self.publish(queue_name, ujson.dumps(body))
  File "./zerver/lib/queue.py", line 108, in publish
    self.ensure_queue(queue_name, do_publish)
  File "./zerver/lib/queue.py", line 88, in ensure_queue
    if not self.connection.is_open:
AttributeError: 'NoneType' object has no attribute 'is_open'

During handling of the above exception, another exception occurred:
[... traceback of connection failure inside the retried self.publish()]
```

That's a type error -- a programming error, not an exceptional
condition from outside the program.  Fix the programming error.

Also move the retry out of the `except:` block, so that if it also
fails we don't get the exceptions stacked on each other.  This is a
new feature of Python 3 which is sometimes indispensable for
debugging, and which surfaced this nit in the logs (on Python 2 we'd
never see the AttributeError part), but in some cases it can cause a
lot of spew if care isn't taken.
2017-11-16 18:33:14 -08:00
Sampriti Panda a696141a25 search: Add more extensive logging for Unicode errors in highlight_string.
Tweaked by tabbott to provide finer-grained logging.
2017-11-16 15:08:16 -08:00
Rhea Parekh 8c4651d716 zerver/lib/actions.py: Fix translation bugs.
Fixes #7396.
2017-11-16 11:05:49 -08:00
Tim Abbott 638eb7a8e4 docs: Update links to ReadTheDocs to always use https.
This is better security practice.

We also add a lint rule to enforce this for the future.
2017-11-16 10:59:24 -08:00