Commit Graph

35832 Commits

Author SHA1 Message Date
Aman Agrawal 1324e7798c ci: Directly run production install script. 2020-04-28 11:11:00 -07:00
Aman Agrawal b5e31c04c9 ci: Rename production install script. 2020-04-28 11:11:00 -07:00
Aman Agrawal d22e26ce26 ci: Rename production build job. 2020-04-28 11:11:00 -07:00
Aman Agrawal 70f7d04bf1 ci: Rename production builder script file. 2020-04-28 11:11:00 -07:00
Steve Howell 1aaef75bc5 test databases: Fix dev/test typo.
This line was changed to the wrong tool by
accident in 23f09fadfa.
2020-04-28 07:13:22 -04:00
Abhishek-Balaji d3db174be9 alert_words: Change test_alert_words to use actions.do_*().
Changing test_alert_words to use do_add_alert_words() and
do_remove_alert_words() from lib/actions.py instead of the
existing add_user_alert_words() and remove_user_alert_words()
as is the general policy of calling these functions when we
are updating the database.
2020-04-27 22:53:25 -07:00
Tim Abbott 976e554799 Revert "home: Don't use pointer to compute furthest_read_time."
This reverts commit 8f32db81a1.

This change unfortunately requires an index that we don't have, and
thus is incredibly expensive.  We'll need to do a thoughtful reworking
before we can integrate it again.
2020-04-27 22:29:39 -07:00
Tim Abbott 1094589f79 test-all: Print a colorful notice on startup discouraging use.
The test-all tool is useful if you want to be really sure everything
passes, but it's almost always better to run a smaller subsuite when
doing active development, and just run test-all (or CI) at the end.

Our docs already covered this issue well, but this presents the idea
in a place one is definitely looking at when running test-all.
2020-04-27 16:35:29 -07:00
Tim Abbott 03fecba917 alert_words: Fix cache flushing behavior and add tests.
The post_init cache-flushing behavior in the original alert words
migration was subtly wrong; while it may have passed tests, it didn't
have the right ordering for unlikely races.

We use post_save rather than post_init hooks precisely because they
ensure that we flush the cache after we know the database has been
updated and any future reads from the database will have the latest
state.
2020-04-27 11:48:32 -07:00
Tim Abbott 8e5b0351b3 alert_words: Fix case-sensitivity of alert words.
Previously, alert words were case-insensitive in practice, by which I
mean the Markdown logic had always been case-insensitive; but the data
model was not, so you could create "duplicate" alert words with the
same words in different cases.  We fix this inconsistency by making
the database model case-insensitive.

I'd prefer to be using the Postgres `citext` extension to have
postgres take care of case-insensitive logic for us, but that requires
installing a postgres extension as root on the postgres server, which
is a pain and perhaps not worth the effort to arrange given that we
can achieve our goals with transaction when adding alert words.

We take advantage of the migrate_alert_words migration we're already
doing for all users to effect this transition.

Fixes #12563.
2020-04-27 11:31:51 -07:00
Abhishek-Balaji 052368bd3e alert_words: Move alert_words from UserProfile to separate model.
Previously, alert words were a JSON list of strings stored in a
TextField on user_profile.  That hacky model reflected the fact that
they were an early prototype feature.

This commit migrates from that to a separate table, 'AlertWord'.  The
new AlertWord has user_profile, word, id and realm(denormalization so
we can provide a nice index for fetching all the alert words in a
realm).

This transition requires moving the logic for flushing the Alert Words
caches to their own independent feature.

Note that this commit should not be cherry-picked without the
following commit, which fixes case-sensitivity issues with Alert Words.
2020-04-27 11:29:50 -07:00
Abhishek-Balaji 818776faae alert bugdown: Change name of AlertWordsNotificationProcessor.
This is a precursor commit to change the name of
AlertWordNotificationProcessor to AlertWordsNotificationProcessor
to match the change from UserProfile.alert_words to Alertword.
2020-04-27 10:45:18 -07:00
Rohitt Vashishtha 087b54aa45 markdown: Keep only 'text' for a no-op pygments rendering.
Previously, we added support for 'none', 'plain' and 'noop' and a
function `lang = remap_language(lang)`. This also had the potential
to encourage adding more remappings- something that we deliberatly
want to keep to a minimum.

For context, Anders K doesn't want us to keep any remapping (only
keeping 'text' which is the default no-op lexer that pygments has)
and Tim wants to keep 'plain' and 'text'. We should only document
and advertise 'text'.
2020-04-27 10:40:29 -07:00
Tim Abbott 2b59590e1f test_unread: Remove test for removed mobile hack.
This should have been included in
731cd79436.
2020-04-26 23:41:53 -07:00
Anders Kaseorg a6fee2f18e notifications: Use electron_bridge.new_notification when available.
This will eventually let us delete a bit of annoying compatibility
code from the desktop app’s injected JavaScript.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-26 23:02:05 -07:00
Anders Kaseorg 549ed3913c notifications: Use NotificationAPI for permission_state.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-26 23:02:05 -07:00
Anders Kaseorg f277eb022c notifications: Use addEventListener to register handlers.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-26 23:02:05 -07:00
Anders Kaseorg cb52f4d7fc notifications: Remove weird Firefox code.
Firefox’s notifications are fine, and iconUrl isn’t a thing.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-26 23:02:05 -07:00
Anders Kaseorg a16cddf84f notifications: Remove cancel_notification_object wrapper.
Running the close handler won’t break anything; it’s safe to delete
from a Map while iterating through it.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-26 23:02:05 -07:00
Anders Kaseorg 0b1c27192f notifications: Remove long-gone webkitNotifications draft API.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-26 23:02:05 -07:00
Anders Kaseorg 8138e06935 notifications: Add link for sending a test notification.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-26 23:02:05 -07:00
Anders Kaseorg e701f20861 electron_bridge: Use getter and setter interface to mutable properties.
This exists in all versions of the desktop app that we still support,
and will eventually let us delete a bit of annoying compatibility code
from the desktop app’s injected JavaScript.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-26 23:02:05 -07:00
clarammdantas 25c5856988 stream_edit: Remove unnacessary conditionals.
The condition was removed because in either case we
want to have the stream_row and not the sub/unsub
button, so we can always get the stream_row directly.
2020-04-26 22:42:38 -07:00
Puneeth Chaganti 8f32db81a1 home: Don't use pointer to compute furthest_read_time.
When a user is reading messages only in stream or topic narrows, the pointer
can be left far behind. Using this to compute the furthest_read_time causes
the banckruptcy banner to be shown even when a user has been actively
reading messages. This commit switches to using the sent time on the last
message that the user has read to compute the furthest read time.
2020-04-26 22:41:52 -07:00
Tim Abbott d8def0de55 pointer: Remove useless last_pointer_updater field.
This hasn't done anything in at least 5 years.
2020-04-26 22:39:24 -07:00
Tim Abbott 731cd79436 messages: Remove first_unread pointer hack.
This hack was important when only the mobile apps (and not the webapp)
were using the unread_msgs data structure and the first_unread
infrastructure.  Now that the webapp is using those things, there
aren't leaked ancient unread messages that aren't accessible on the
webapp, so any few users still in this situation can get out of it by
just reading the problematic messages.
2020-04-26 22:37:41 -07:00
Tim Abbott 06f3664cde pointer: Remove unread debugging management commands.
I don't think we've had a use for these tools since our unread systems
stabilized shortly after they were written, so it makes sense to just
remove them rather than updating them for the pointer migration.
2020-04-26 22:37:41 -07:00
Tim Abbott 69ae4931c3 migrations: Use django.db.backends.postgresql.schema.
This replaces django.db.backends.postgresql_psycopg2, which has been
an alias to django.db.backends.postgresql since Django 1.9.
2020-04-26 22:20:24 -07:00
Abhishek-Balaji 2ff1527be8 models: Switch from NullBooleanField to BooleanField.
In Django 2.1, the preferred way to express a nullable BooleanField
changed from NullBooleanField to passing null=True to BooleanField.

This updates our codebase to use the preferred API.  Tweaked by
tabbott to update the linter rules.

The migration is a noop for Django accounting only.

Part of #11341.
2020-04-26 22:13:28 -07:00
Steve Howell c11de1e3a7 user profile cards: Fix bug with deactivated user.
We had a user have problems with the user
profile menus that you get when you click
on either sender avatars or mention pills.

If a deactivated user had a long enough email
that we would normally want to un-hide the clipboard
icon for them, we would crash inside of
`init_email_clipboard`, because the icon isn't
there for them.  If the user didn't have the
console open to see the crash, the symptom
became that you would get multiple cards
visible and kind of "stuck".

I chose to fix this by just making the code
defensive with respect to the absence of the
icon, instead of short-circuiting it for
deactivated users.

It's a bit odd that we still have an element
matching `.user_email_popover` in the profile
card for deactivated users, since that element
doesn't actually include an email, but it instead
says "(This user has been deactivated)".  I
considered removing the class, but the CSS
that we use for emails kind of applies here
too.

Testing this is a kind of a pain, as you want
either long emails or to just temporarily hack
this condition to true:

    if (this.clientWidth < this.scrollWidth) {
        // ...
    }

And then test with a deactivated user, of course.

Fixes #14473
2020-04-26 11:12:45 -07:00
Mateusz Mandera 3b582931af migration: Migrate messages tied to removed bots.
This cleans up any messages that might have been exchanged with
`NEW_USER_BOT` or `FEEDBACK_BOT` (cross-realm bots that were last
used, as far as we know, years ago) that have been completely removed
from the codebase.

Details on the algorithm are in the migration code itself.

Fixes #13583.
2020-04-26 11:08:39 -07:00
Tim Abbott 9566905b81 docs: Clarify nginx reverse proxy documentation.
Fixes #14740.
2020-04-26 10:51:39 -07:00
rebtung f7fbe3419f api: Improve consistency of reactions API.
Previously, the message and event APIs represented the user differently
for the same reaction data. To make this more consistent, I added a
user_id field to the reaction dict for both messages and events. I
updated the front end to use the user_id field rather than the user
dict. Lastly, I updated front end and back end tests that used user
info.

I primarily tested this by running my local Zulip build and
adding/removing reactions from messages.

Fixes #12049.
2020-04-26 10:35:55 -07:00
Puneeth Chaganti 2a65be2bf5 url preview: Use Chrome's user agent instead of a Zulip one.
Some sites don't render correctly unless you are one of the latest browsers.
YouTube Music, for instance, changes the page title to "Your browser is
deprecated, please upgrade.", which makes our URL previews look bad.
2020-04-26 10:16:43 -07:00
Aman Agrawal 8a841a5634 ci: Update bionic docker images to save upgrade time in builds. 2020-04-25 16:10:55 -07:00
Aman Agrawal cf14cc211f ci: Separate prod tarball extraction and install process.
This allows to run scripts between extraction and install
process.

It will be used to restore npm caches for production install jobs.

We extract the tarball in the working directory so that yarn.lock and
package.json are available to restore cache.

(And also so the path is deterministic).
2020-04-25 16:10:41 -07:00
Aman Agrawal 0aa18e88a5 ci: Stop cloning repo as it is not required for prod install.
Move prod installer and test files to persistent dir so that
they can be utilized.
2020-04-25 16:07:55 -07:00
Aman Agrawal a015160ec0 ci: Fix restoring and and saving npm caches. 2020-04-25 16:07:48 -07:00
Tim Abbott 5e24d26f45 webpack: Remove useless print statement.
In I think every place we run this, you'll already be able to see
`tools/webpack` in the `set -x` style output anyway.
2020-04-25 15:39:22 -07:00
Tim Abbott 5187d5032c update-prod-static: Remove unused authors-not-required.
This argument hasn't done anything since we moved constructing the
/team page to a cron job.
2020-04-25 15:39:03 -07:00
Tim Abbott 7e0eeb20a3 update-prod-static: Stop hiding output in update-prod-static.log.
Now that we've cleaned up this tool's output, there's no reason to use
an awkward mechanism to hide its output; we can just print it out like
a normal program.

Fixes #14644; resolves #14701.
2020-04-25 15:39:03 -07:00
Tim Abbott c880886cc3 update-prod-static: Reduce verbosity of logging.
Previously, update-prod-static would produce thousands of lines of
basically useless output in `var/log/update-prod-static.log`.
2020-04-25 15:39:03 -07:00
Anders Kaseorg b46d1c8d07 dev_settings: Fix run-dev SyntaxError.
zthumbor loads dev_settings from Python 2.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-24 20:08:01 -07:00
pemontto 01a57ddd9d webhooks: Use Gitlab project web_url over repo homepage.
This is preferred because the `repository` field isn't available for
project-level events.
2020-04-24 17:50:47 -07:00
Puneeth Chaganti e41f4b01c0 requirements: Upgrade IPython to the latest version. 2020-04-24 17:25:33 -07:00
Tim Abbott 96443c0a1d settings: Fix missing typing import.
The world previously passed CI only because old versions of
prod_settings_template have this already present via the * import.
2020-04-24 17:08:01 -07:00
Wyatt Hoodes 7f637f7cb1 migrations: Deprecate migration 0077 into a noop.
This ancient migration imports boto, which interferes
with our upgrade to boto3.

> git name-rev f13d6a18eb
f13d6a18eb tags/1.6.0~1082

We can safely assume nobody is upgrading from a server on <1.6.0,
since we have no supported platforms in common with those releases.
2020-04-24 17:03:14 -07:00
Wyatt Hoodes 6d686d0e4b migrations: Deprecate migration 0064 into a noop.
This ancient migration imports boto, which interferes
with our upgrade to boto3.

> git name-rev 8ae35211b5
8ae35211b5 tags/1.6.0~1924

We can safely assume nobody is upgrading from a server on <1.6.0,
since we have no supported platforms in common with those releases.
2020-04-24 17:03:14 -07:00
Wyatt Hoodes 3e8d877564 migrations: Deprecate migration 0060 into a noop.
This ancient migration imports boto, which interferes
with our upgrade to boto3.

> git name-rev a32f666f5c
a32f666f5c tags/1.6.0~2384

We can safely assume nobody is running servers on <1.6.0; there are no
supported platforms in common with 1.6.0 anyway.
2020-04-24 17:03:14 -07:00
Anders Kaseorg c0ffa71fa9 nginx: Replace unanchored regexes in location directives.
We could anchor the regexes, but there’s no need for the power (and
responsibility) of regexes here.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-24 16:58:19 -07:00