Commit Graph

4467 Commits

Author SHA1 Message Date
Conner Bondurant 4d4b6b8319 email: Convert send_email to use formataddr.
Changed the requirements for UserProfile in order to allow use of
the formataddr function in send_mail.py.

Converted send_email to use formataddr in conjunction with the commit
that strengthened requirements for full_name, such that they can now be
used in the to field of emails.

Fixes #4676.
2019-07-22 18:15:58 -07:00
Conner Bondurant c25dcf048d models: Enforce stricter requirements on the full_name field.
This changes the requirements for UserProfile to disallow some
additional characters, with the overall goal of being able to use
formataddr in send_mail.py.

We don't need to be particularly careful in the database migration,
because user full_names are not required to be unique.
2019-07-22 18:13:34 -07:00
neiljp (Neil Pilgrim) ba7a0934e3 requirements: Upgrade mypy to 0.711.
This comes with it a big performance improvement; mypy is now only
barely our slowest linter even if it wasn't previously running.

Fixes: #12058
2019-07-22 17:12:50 -07:00
neiljp (Neil Pilgrim) 80a4504784 mypy: Clean up typing for CustomProfileField code. 2019-07-22 17:10:50 -07:00
Wyatt Hoodes e4133e687f test_runner: Ignore inheritance issues in TestSuite.
Again, `result` of type `TestResult` is considered by mypy to
not have these _attributes acquired through inheritance. So,
we ignore them.
2019-07-22 17:06:41 -07:00
Wyatt Hoodes 5686821150 middleware: Change write_log_line to publish as a dict.
We were seeing errors when pubishing typical events in the form of
`Dict[str, Any]` as the expected type to be a `Union`.  So we instead
change the only non-dictionary call, to pass a dict instead of `str`.
2019-07-22 17:06:41 -07:00
Wyatt Hoodes dd19e2df1b test_runner: Ignore self.stream in TextTestResult class.
Per the import line:

`from unittest import loader, runner  # type: ignore  # Mypy cannot pick
these up.`

Because `TextTestResult` inherits from `runner.TextTestResult`, mypy
doesn't see `self` as having an attribute `stream`, so we ignore these
instead of cluttering with `casts` or `isinstances`.
2019-07-22 16:45:21 -07:00
Tim Abbott d3f59db8ce test_runner: Cleanup typing for TestResult objects.
There's two possible types here.
2019-07-22 16:45:21 -07:00
Wyatt Hoodes 76e8dc1657 test_runner: Change type to unittest.TestSuite.
In `deserialize_suite`, it passes a test as being of the
super type, where just having TestSuite assumes the type of
`zerver.Runner.TestSuite`.
2019-07-22 16:27:39 -07:00
Wyatt Hoodes 3509fe07b5 test_runner: Fix incorrect type for enforce_timely_test_completion.
Our TestResult objects are always actually TextTestResults.
2019-07-22 16:27:39 -07:00
Tim Abbott 02d55928ea import: Fix importing slack avatars into S3_UPLOAD_BACKEND.
Apparently, a subtle mismatch between the filename/URL formats for our
upload codebases meant that importing Slack avatars into systems using
S3_UPLOAD_BACKEND would end up with the avatars having the wrong URLs.
2019-07-21 21:25:31 -07:00
Hemanth V. Alluri d62b75fc4c openapi: Remove /users/me/subscriptions from pending_endpoints.
In commit 7c71e98, we added a special exception for the
/users/me/subscriptions endpoint in the automatic validation test.
By adding some extra documentation, we now remove this extra code,
as well as the endpoint from the list of pending endpoints.
2019-07-21 18:39:53 -07:00
Rishi Gupta 1a6f8cdf17 emails: Remove unsubscribe link from missed message emails.
This allows the footer to be two lines long instead of three.
2019-07-21 15:10:48 -07:00
Tim Abbott 21c8a7ec36 i18n: Fix strings in send_pm_if_empty_stream.
The previous iteration did not properly handle languages with a
different word order than English.

Discovered via warning output in `manage.py makemessages`.
2019-07-21 14:43:50 -07:00
Mateusz Mandera f1b135bd16 email_mirror: Rename include-quotations to include-quotes. 2019-07-20 15:53:43 -07:00
Mateusz Mandera 519ed41803 email_mirror: Display stream email address with . as the separator. 2019-07-20 15:53:43 -07:00
Mateusz Mandera 8bbd1d005d email_mirror: Support both + and . as separators in the address. 2019-07-20 15:53:42 -07:00
Wyatt Hoodes e331a758c3 python: Migrate open statements to use with.
This is low priority, but it's nice to be consistently using the best
practice pattern.

Fixes: #12419.
2019-07-20 15:48:52 -07:00
Mateusz Mandera 842d9e944a test_classes: Prepare for moving system bots to zulipinternal. 2019-07-20 14:55:21 -07:00
Rishi Gupta 38eeb3e183 notification bot: Add constant for STREAM_EVENTS_NOTIFICATION_TOPIC. 2019-07-20 14:39:15 -07:00
Rishi Gupta c9f194be99 notification bot: Update message for stream name changes.
Also properly internationalizes string formatting arguments for the stream
created message.
2019-07-20 14:39:15 -07:00
Rohitt Vashishtha 726d5003e1 bugdown: Force absolute urls in topic links.
If a url doesn't have a scheme, browsers would treat it as a relative
url and open something like: https://chat.zulip.org/google.com instead.

This PR fixes the issue on the backend; the frontend implementation
remains out of sync and the user sending the message wouldn't see
any linkification for urls without a scheme.

Fixes #12791.
2019-07-19 12:02:52 -07:00
Anders Kaseorg afa251de5d dev_urls: Serve static files with staticfiles_urlpatterns.
The test_docs change is because Django runs test cases with DEBUG =
False, which ordinarily means it doesn’t serve /static during tests.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-19 11:47:51 -07:00
Anders Kaseorg a7eb3faf96 storage: Move the staticfiles.json hack to ZulipStorage.
There’s no reason to monkey-patch something that we were already
subclassing.

Removing the PRODUCTION conditional causes us to generate
staticfiles.json in the right place to begin with so we don’t need to
move it later.  It also allows Django to find staticfiles.json if
running the dev server with PIPELINE_ENABLED.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-19 11:47:51 -07:00
Rishi Gupta bd52ec9f95 notification bot: Remove checks for whether bot exists. 2019-07-17 17:54:16 -07:00
Rishi Gupta f4397166c4 notify_new_user: Remove call to statsd.gauge.
This has been deprecated by our analytics system.
2019-07-17 17:54:16 -07:00
Rishi Gupta b520df18ee notify_new_user: Move clear_scheduled_invitation_emails out of function.
notify_new_user is confusingly named, but it should be read as "notify
others that a new user was created", not "notify the new user".
2019-07-17 17:54:16 -07:00
Rishi Gupta c030411fec email visibility: Use delivery_email to clear scheduled invitation emails.
This fixes a user-visible bug, where users signing up for realms with
restricted email visibility get reminder emails 1 week later, whether or not
they created an account.
2019-07-17 17:54:16 -07:00
Vishnu Ks 3a6fc09a4d emails: Make sender name go in-line with message body.
Making sender name go in-line with message body only if
the html starts with <p> tag since it won't look good
if the message starts with a code snippet, ul, etc.
If message starts with p tag we can safely assume that
it can go in-line with sender name.
2019-07-17 17:50:04 -07:00
Priyank Patel c067c155aa messages: Support passing user ID to sender operator. 2019-07-17 16:09:12 -07:00
Wyatt Hoodes c218735b2c test_runner: Add error handling for test directory clean up.
It was discovered that errors such as:
    `OSError: [Errno 16] Device or resource busy`
potentially arise when running in serial mode, or with
explicit test cases passed to `test-backend`.
2019-07-15 14:15:19 -07:00
Tim Abbott 7c71e987ac openapi: Fix endpoints incorrectly tagged as documentation_pending. 2019-07-15 13:39:29 -07:00
Tim Abbott 588a24b661 cache: Fix caching of system bots.
In the unlikely event that someone edited the properties of a system
bot and then saved the result, we were still caching the old version
indefinitely in the get_system_bot cache.

This led to a confusing case where a newly installed Zulip server
didn't have is_api_super_user properly set on its EMAIL_GATEWAY_BOT in
memcached.

Co-authored-by: Mateusz Mandera <mateusz.mandera@protonmail.com>
2019-07-15 12:32:29 -07:00
Rishi Gupta 15cd2903dd notification bot: Refactor arguments to send_pm_if_empty_stream.
Previously we sent "" for stream_name where we should have sent None, which
made this function harder to understand. The "" value was never used.

This also reorders the arguments to be match the order of the arguments in
the two callers.
2019-07-13 15:53:36 -07:00
Rishi Gupta 628d9ad67d notification bot: Update error messages in send_pm_if_empty_stream. 2019-07-13 15:53:36 -07:00
Rishi Gupta b92f936dad notification bot: Refactor send_pm_if_empty_stream to simplify logic. 2019-07-13 15:53:36 -07:00
David Wood 9bace3f2cd notifications: Allow only notifiable in unread count.
This commit adds a new setting to the user's notification settings that
will change the behaviour of the unread count in the title bar and
desktop application.

When enabled, the title bar will show the count of unread private messages
and mentions. When disabled, the title bar will act as before, showing
the total number of unread messages.

Fixes #1736.
2019-07-13 15:49:04 -07:00
Hariom Verma 107da5402c url preview: Replace YouTube URLs with their titles.
Modified by punchagan to:
* Replace URLs with titles only if the inline url embed previews are turned on
* Add a test for youtube titles replacing URLs

The titles for the videos are fetched asynchronously after the message has been
sent via the code that fetches metadata for open graph previews. So, the URLs
are replaced with titles only if the inline embed url previews feature is
enabled.

Ideally, YouTube previews should be shown only if inline url previews are
enabled, but this feature is in beta, while YouTube previews are pretty stable.
Once this feature is out of beta, YouTube previews should be shown only if the
url previews feature is turned on.

YouTube preview image is calculated as soon as the message is sent, while the
title needs to be fetched using a network request. This means that the URL is
replaced only after the data has been fetched from the request, and happens a
couple of seconds after the message has been rendered.

Closes #7549
2019-07-12 19:14:19 -07:00
Puneeth Chaganti 865bc24f67 url preview: Avoid showing previews for URLs in blockquotes.
Messages with links embedded in blockquotes turn out to be replies to
messages with links, more often than not. Showing previews for links in
replies seems like clutter, and it seems reasonable to turn off previews for
such links.
2019-07-12 19:14:00 -07:00
Puneeth Chaganti ad98f536bf bugdown: Fix incorrect type annotation for ElementPair. 2019-07-12 19:14:00 -07:00
Wyatt Hoodes 0e3fddbc6e test_fixtures: Add logic for removing stale test directories.
Similarly to how stale database removal is handled, we add a check for
stale test run directories at the end of the `test-backend` script.
2019-07-12 17:42:18 -07:00
Wyatt Hoodes 62566f02d7 tests: Use TEST_WORKER_DIR for file access. 2019-07-12 17:42:18 -07:00
Wyatt Hoodes 5f20caa6e0 test_upload: Refactor test_upload output to new filepath.
We write a function to set the `LOCAL_UPLOADS_DIR` path depending
on whether tests are being run in parallel or serial mode.
2019-07-12 17:35:44 -07:00
Wyatt Hoodes ff156c1f8a test_runner: Create file structure for test-backend filepath rework.
We create a path structure in the from:
 `/var/<uuid>/test-backend/run_1234567/worker_N/`

A settings attribute, `TEST_WORKER_DIR`, was created as a worker's
directory for a given `test-backend` run's file storage.  The
appropirate path is created in `setup_test_environment`, while each
workers subdirectory is created within `init_worker`.

This allows a test class to write to `settings.TEST_WORKER_DIR`,
populating the appropirate directory of a given worker.  Also
providing the long-term approach to clean up filesystem access
in the backend unit tests.
2019-07-12 17:27:18 -07:00
Rohitt Vashishtha 0ba332bcc0 topic-mention: Add Bugdown implementation as StreamTopicPattern. 2019-07-11 14:53:10 -07:00
Rishi Gupta 5400eb62be notification bot: Inline prep_stream_welcome_message.
This will make it easier to have access to the stream creator.

The indirection also isn't really adding anything, especially given that the
announce message is inlined just above.
2019-07-11 14:16:56 -07:00
Rishi Gupta 7de7b6872b notification bot: Modify initial stream creation message.
Discussion at
https://chat.zulip.org/#narrow/stream/137-feedback/topic/hello.20topic

The "by @**X**" part in the message content is in a later commit.
2019-07-11 14:16:56 -07:00
Aayush Agrawal 54584f6c16 url preview: Create a single preview for each URL in a message.
Modified by punchagan to:
* Add a separate markdown test for de-duplicating inline previews
* Check for number of unique URLs to see if per limit message is crossed
* Use a set for processed URLs instead of a list

Fixes #8379.
2019-07-11 13:37:15 -07:00
Hemanth V. Alluri ffd2bccd4e openapi: Standardize URL format to use {var_name} syntax.
The previous code for the validator test was fairly messy due to
checking for both formats of the openapi url, one with
<variable_name> and the other with {variable_name}. To eliminate
this, we have standardized the format and restricted it to
{variable_name} as per the official format at:
https://swagger.io/docs/specification/describing-parameters.
2019-07-11 11:01:40 -07:00
Wyatt Hoodes 0f31053b0c export.py: Call upload_backend only when we know which backend to use.
Importing at the top of the file causes conflictions on the
backend when deciding whether to go down the S3 path, or the
`LOCAL_UPLOADS_DIR` path.
2019-07-10 17:48:54 -07:00