Commit Graph

4359 Commits

Author SHA1 Message Date
Vishnu KS d6743da139 docs test: Verify zulip/zulip file and directory links locally. 2020-11-05 10:07:43 -08:00
Vishnu KS cba292b4bb lint: Ensure that docs don't link directly to code lines. 2020-11-05 10:07:43 -08:00
YashRE42 967efc32d2 widgets: Remove tictactoe example widget.
Steve asked me to remove this, since the tictactoe game was always
intended as a proof of concept. Now that we have poll and todo
widgets, the sample code for tictactoe has much less value.

We replace the content and type in test_widgets.py to maintain
coverage.
2020-11-03 14:46:39 -08:00
Aman Agrawal 87cdd8433d home: Allow logged out user through home.
We allow user to load webapp without log-in. This is only
be enabled for developed purposes now. Production setups will
see no changes.
2020-11-02 17:07:12 -08:00
Vishnu KS 7a6285ede7 email testing: Store the SMTP settings in dev-secrets.
It's super annoying to set this up each time I have to test
email templates in gmail.
2020-10-30 11:50:30 -07:00
Anders Kaseorg cc55393671 python: Open text files as text to skip decode operations.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-30 11:36:38 -07:00
Anders Kaseorg aaa7b766d8 python: Use universal_newlines to get str from subprocess.
We can replace ‘universal_newlines’ with ‘text’ when we bump our
minimum Python version to 3.7.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-30 11:36:38 -07:00
Vishnu KS d4f5cea559 docs: Mark chat.zulip.org as an external link for testing.
The comment explains the background, but basically there's no purpose
in trying to verify this via CI.
2020-10-29 16:42:17 -07:00
Alex Vandiver c1201bd69c provision: Rename variables to postgresql. 2020-10-28 11:57:03 -07:00
Alex Vandiver 2b0bbbb882 tools: Rename postgres to postgresql in tool names. 2020-10-28 11:57:02 -07:00
Alex Vandiver 5eb8064a1a install: Rename postgres options to postgresql. 2020-10-28 11:55:32 -07:00
Alex Vandiver 1f7132f50d docs: Standardize on PostgreSQL, not Postgres. 2020-10-28 11:55:16 -07:00
Alex Vandiver e124324050 puppet: Rename postgres_appdb in nagios to postgresql. 2020-10-28 11:51:52 -07:00
Anders Kaseorg 4e9d587535 python: Pass query parameters as a dict when making GET requests.
This provides automatic URL-encoding.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-27 13:47:02 -07:00
Alex Vandiver f0b11d62f6 provisioning: Support non-RSA SSH keys.
Use the parts of the JSON to determine the files it should create.
2020-10-27 13:43:45 -07:00
Alex Vandiver 7b2112baf3 lint: Add linters to help enforce profile/base layering. 2020-10-27 13:29:19 -07:00
Alex Vandiver 188af57296 puppet: Rename postgres_appdb to postgresql.
There is only one PostgreSQL database; the "appdb" is irrelevant.
Also use "postgresql," as it is the name of the software, whereas
"postgres" the name of the binary and colloquial name.  This is minor
cleanup, but enabled by the other renames in the previous commit.
2020-10-27 13:29:19 -07:00
Alex Vandiver c2185a81d6 puppet: Move top-level zulip deployments into "profile" directory.
This moves the puppet configuration closer to the "roles and profiles
method"[1] which is suggested for organizing puppet classes.  Notably,
here it makes clear which classes are meant to be able to stand alone
as deployments.

Shims are left behind at the previous names, for compatibility with
existing `zulip.conf` files when upgrading.

[1] https://puppet.com/docs/pe/2019.8/the_roles_and_profiles_method
2020-10-27 13:29:19 -07:00
Anders Kaseorg 31d0141a30 python: Close opened files.
Fixes various instances of ‘ResourceWarning: unclosed file’ with
python -Wd.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-26 12:31:30 -07:00
Anders Kaseorg 72d6ff3c3b docs: Fix more capitalization issues.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-23 11:46:55 -07:00
Alex Vandiver 50e9e2ed20 puppet: Make zulip::base include zulip::apt_repository.
There was likely more dependency complexity prior to 97766102df, but
there is now no reason to require that consumers explicitly include
zulip::apt_repository.
2020-10-22 11:30:53 -07:00
Anders Kaseorg 5c6202ca97 notifications: Use a dynamic SVG favicon for unread counts.
Closes #2304.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-21 07:43:00 -07:00
Anders Kaseorg 3e8e4b9cad check-thirdparty: Work around deficient Mypy annotations for debian.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-19 11:54:14 -07:00
YashRE42 e344ea25f8 landing-page: Rename to landing_page.css. 2020-10-19 11:50:39 -07:00
YashRE42 7ee3484b7b portico-styles: Rename to portico_styles.css. 2020-10-19 11:50:39 -07:00
Anders Kaseorg 0be17b6b00 semgrep: Treat ugettext_lazy like ugettext.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-18 14:31:15 -07:00
Mateusz Mandera 716df658fa queue_processors: Don't run test queues with run-dev.py. 2020-10-18 14:07:31 -07:00
Mateusz Mandera 269faa342a test-queue-worker-reload: Fix expected worker count.
We didn't bump this when adding some additional test queues, that run
with run-dev.py as of this commit.
2020-10-18 14:07:31 -07:00
Aman Agrawal bdc4721aee BaseDocumentationSpider: Don't crawl webapp. 2020-10-16 16:26:29 -07:00
Aman Agrawal 72b25553b2 BaseDocumentationSpider: Print correct link on error. 2020-10-16 16:23:33 -07:00
Anders Kaseorg 6099612a15 lint: Add shfmt as a linter.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-15 15:16:00 -07:00
Anders Kaseorg dfaea9df65 shfmt: Reformat shell scripts with shfmt.
https://github.com/mvdan/sh

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-15 15:16:00 -07:00
Alex Vandiver 0d5760d59f install-aws-server: Force non-interactive dist-upgrade.
Installing an updated linux kernel package, as can happen during the
`apt dist-upgrade` done by the installer, can cause grub to pop up a
prompt to update its configuration file.  In an unattended headless
configuration, this will stop the installation.

Explicitly configure apt to be non-interactive, and prefer the newest
configuration, during the install.
2020-10-15 14:39:20 -07:00
Steve Howell a9356508ca events: Stop sending occupy/vacate events.
We used to send occupy/vacate events when
either the first person entered a stream
or the last person exited.

It appears that our two main apps have never
looked at these events.  Instead, it's
generally the case that clients handle
events related to stream creation/deactivation
and subscribe/unsubscribe.

Note that we removed the apply_events code
related to these events.  This doesn't affect
the webapp, because the webapp doesn't care
about the "streams" field in do_events_register.

There is a theoretical situation where a
third party client could be the victim of
a race where the "streams" data includes
a stream where the last subscriber has left.
I suspect in most of those situations it
will be harmless, or possibly even helpful
to the extent that they'll learn about
streams that are in a "quasi" state where
they're activated but not occupied.

We could try to patch apply_event to
detect when subscriptions get added
or removed. Or we could just make the
"streams" piece of do_events_register
not care about occupy/vacate semantics.
I favor the latter, since it might
actually be what users what, and it will
also simplify the code and improve
performance.
2020-10-14 10:53:10 -07:00
Anders Kaseorg dd48dbd912 docs: Add spaces to “check out”, “log in”, “set up”, “sign up” as verbs.
“Checkout”, “login”, “setup”, and “signup” are nouns, not verbs.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-13 15:47:13 -07:00
Steve Howell 9df9934ed6 refactor: Pass realm to bulk_add_subscriptions.
I think it's important that the callers understand
that bulk_add_subscriptions assumes all streams
are being created within a single realm, so I make
it an explicit parameter.

This may be overkill--I would also be happy if we
just included the assertions from this commit.
2020-10-13 11:28:17 -07:00
Anders Kaseorg 7f69c1d3d5 python: Catch specific exceptions from requests.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-11 16:11:41 -07:00
Anders Kaseorg aabef3d9be python: Catch specific exceptions from orjson.
Followup to #16120.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-11 16:11:41 -07:00
Anders Kaseorg a109d68af0 create-test-api-docs: Remove unhelpful exception handler.
encode returns bytes, which cannot possibly be written to a text file.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-11 16:11:35 -07:00
Anders Kaseorg f81a5e87ed run-dev: Wait for children to exit on Ctrl+C after killing them.
In addition to being generally more correct, this works around a bug
in Node.js that causes webpack-dev-server to corrupt the terminal
state when exiting as a background process.

https://github.com/nodejs/node/issues/35536

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-07 16:15:19 -07:00
Anders Kaseorg 1c7db528bd test_server: Wait for run-dev after terminating it.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-07 16:15:19 -07:00
Anders Kaseorg f92224c744 eslint: Fix unicorn/catch-error-name.
https://github.com/sindresorhus/eslint-plugin-unicorn/blob/master/docs/rules/catch-error-name.md

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-07 16:00:33 -07:00
Alex Vandiver 150d782d6b production-verify: Fix puppet path. 2020-10-06 17:26:14 -07:00
Alex Vandiver 8cf37a0d4b queue: Add a tool to profile no-op enqueue and dequeue actions. 2020-10-06 17:26:14 -07:00
Alex Vandiver 3e6cc49ab5 lint: Add missing spaces at line wraps, remove periods. 2020-10-04 17:57:55 -07:00
Anders Kaseorg cf5ededa35 openapi: Use at most one inline subschema in allOf.
This fixes some of the warnings from openapi-generator.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-30 19:45:28 -07:00
Anders Kaseorg f644bf2063 webpack: Use Webpack’s mode option rather than repurposing env.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-28 10:57:08 -07:00
Tim Abbott 94a9fa1891 event_schema: Add documentation and rename a few functions.
This should help make this revised subsystem readable for more new
contributors.  We still need to make updates to the high-level
documentation.
2020-09-25 12:53:00 -07:00
Steve Howell f6e0171d02 event_schema: Split check_reaction into add/remove.
It happens that whether you add a reaction or remove
a reaction, we send the exact same fields, just using
a different op code.

This sort of symmetry is actually kind of rare, as
usually "add" events have more fields, and "remove" events
might just send an id of something to remove.

Our openapi schema treats these as two seperate events,
so we are more consistent with it, and it helps our
schema-checking tooling for node fixtures, too.

Note that we now have to exempt the two events from
our openapi checks, due to the is_mirror_dummy field
in the deprecated user block.  We can decide how to
handle this later--one possibility is to just add it
as an optional field on the event_schema side.
2020-09-25 11:43:20 -07:00
Steve Howell 0c4286222f event_schema: Extract check_realm_update_dict. 2020-09-25 11:43:20 -07:00
Steve Howell 4084f0b949 event_schema: Extract check_realm_user_add.
Note that we make the schema for profile_data
slightly more realistic, but it doesn't actually get
exercised by our current tests (apart from
making sure it's a dict), since we don't have
profile data for our test realm.

We also don't have the optional fields for bots,
since our tests don't exercise that, nor
delivery_email.

So we exempt realm_user_add_event from openapi
checks for now.

When we try to match the openapi specs better, we
will probably want to add a few tests to test_events.

Obviously getting good coverage for adding users
would be nice for all these scenarios:

    * delivery_email matters
    * bots
    * realm has profile fields
2020-09-25 11:43:19 -07:00
Steve Howell 6c74a44697 data_types: Generalize StringDictType.
This is a prep commit for supporting "presence"
events, where the key of the dictionary is some
arbitrary string like "website" but the value
of the dictionary is another dictionary itself
with keys that are more like variable names.
2020-09-25 11:43:19 -07:00
Steve Howell 4f3d5f2d87 event_schema: Extract check_realm_filters.
We have some known issues with representing
tuples in openapi, so we exempt realm_filters
from the relevant check.
2020-09-25 11:43:19 -07:00
Steve Howell e40a5400e5 event_schema: Extract check_muted_topics.
This also forces us to create TupleType.

We exempt this from the openapi check,
since we haven't figured out how to model
tuples in openapi with the same precision
as event_schema (and it may be impossible).

Long term we just want to stop dealing in
tuples, of course.
2020-09-25 11:43:19 -07:00
orientor 91ca1afe98 data_type: Add StringDict data type.
StringDict is a data type for representing dictionaries where
all keys and values are strings. Add this data type to data_types.py
and edit other files so that this data type is put to use and tested.

(slightly tweaked by @showell to remove a comment and shorten
a var name now that we have a proper data type)
2020-09-25 11:43:19 -07:00
Steve Howell d28c01284c event_schema: Extract check_hotspots.
This forces us to introduce a NumberType.
2020-09-25 11:43:19 -07:00
Steve Howell cf26151cea event_schema: Use realm_user_person_types.
For realm_user events, we now structure the
person type as a union of dicts, which is
more consistent with how we model this in
our openapi spec.
2020-09-25 11:43:19 -07:00
Steve Howell 10952394b0 test_events: Use int value of message_retention_days.
We also make our schema in event_schema reflect this,
which in turn makes us match the already accurate
openapi spec, so we no longer need to exempt four
types of events from our sanity checks.
2020-09-25 11:43:19 -07:00
Steve Howell 73e7f7edec check-node-fixtures: Compare python/openapi schemas.
We might want to rename the tool to something more
general now, since we are really reconciling three
things:

    - node fixtures
    - event_schema checkers for test_events
    - openapi specs

The way we compare python and openapi schemas is
as follows:

    - first convert openapi schemas to be build
      from DictType, ListType, etc. with from_opeapi

    - do a diff on the schemas

Most of the new code is just having the FooType
family of classes serialize themselves with schema().
2020-09-25 11:43:19 -07:00
Steve Howell aca641a4d1 refactor: Extract data_types module.
Defining types with an object hierarchy
of type classes will allow us to build
functionality that was impossible (or
really janky) with the validators.py
approach of composing functions.

Most of the changes to event_schema.py
were automated search/replaces.

This patch doesn't really yet take
advantage of the new FooType classes,
but we will use it soon to audit our
openapi specs.
2020-09-25 11:43:19 -07:00
Anders Kaseorg 3a39c6f67f dependencies: Downgrade simplebar to 5.2.1.
SimpleBar 6.0.0-beta.2 through -beta.6 are built with ES6 syntax (I
assume inadvertently: https://github.com/Grsmto/simplebar/issues/523),
and its latest tag has moved back to 5.2.1 anyway.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-24 16:13:47 -07:00
Aman Agrawal 3ec23e1a9d fetch_initial_state_data: Handle case of web public guests.
user_profile will be None for web_public_guests here.  Hence, for
settings (of which most be inaccessible by web public guest),
which require a user_profile, we either set an empty value for
them or set them to a default value. This will help render
the frontend or extend support to our clients without breaking
a lot of code.

Tweaked by tabbott to add many comments.
2020-09-23 12:11:15 -07:00
Anders Kaseorg 5216b70691 lint: Remove custom rule already enforced by Prettier.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-22 18:11:21 -07:00
Mateusz Mandera cd9b194d88 queue: Eliminate useless "burst" concept in monitoring.
The reason higher expected_time_to_clear_backlog were allowed for queues
during "bursts" was, in simpler terms, because those queues to which
this happens, intrinsically have a higher acceptable "time until cleared"
for new events. E.g. digests_email, where it's completely fine to take a
long time to send them out after putting in the queue. And that's
already configurable without a normal/burst distinction.
Thanks to this we can remove a bunch of overly complicated, and
ultimately useless, logic.
2020-09-21 15:24:04 -07:00
Mateusz Mandera 2365a53496 queue: Fix a race condition in monitoring after queue stops being idle.
The race condition is described in the comment block removed by this
commit. This leaves room for another, remaining race condition
that should be virtually impossible, but nevertheless it seems
worthwhile to have it documented in the code, so we put a new comment
describing it.
As a final note, this is not a new race condition,
it was hypothetically possible with the old code as well.
2020-09-21 15:22:56 -07:00
Sumanth V Rao 4a2791e2a3 tools/build_pygments_data: Map pygments language aliases to lexer name.
We need this information in the frontend to:
    * Display the 'view in playground' option for locally echoed messages.
    * When we add a UI settings for realm admins to configure their
      playground choices, we'll need to use these canonicalized aliases
      for displaying the option.

Hence, this tweaks the tool which generates pygments_data.json to contain
the data we need.

Bumping major PROVISION_VERSION since folks need to provision in both
directions.

Tests amended.
2020-09-18 17:12:26 -07:00
Alex Vandiver ece0aaa6cc tests: Remove use_db flag from test_server_running.
It was only ever true, since 2563f93d46.
2020-09-18 15:13:40 -07:00
Alex Vandiver 922940e223 tornado: Be more explicit about tornado configuration in testing.
Passing this through the environment needlessly complicates the logic.
2020-09-18 15:13:40 -07:00
Tim Abbott 9c0d6becc5 docs: Add comment links to i18n documentation. 2020-09-18 11:44:04 -07:00
Aman Agrawal d9814e5e1d recent_topics: Test deleting messages not stored locally.
This commits adds test coverage for
5142bdd317.
2020-09-15 17:03:27 -07:00
Anders Kaseorg b00a0bfe53 js: Use simplebar ES module.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-15 16:33:28 -07:00
Anders Kaseorg a3d26d701e styles: Rename .scss files back to .css.
css-loader@4 broke @import statements referencing files with
extensions other than .css, unless those @import statements are
compiled away by another loader.  Upstream is more interested in
arguing that such @import statements are semantically incorrect than
applying the one line fix.

https://github.com/webpack-contrib/css-loader/issues/1164

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-15 16:33:28 -07:00
Anders Kaseorg 2acc3fbe14 lint: Use prettier --loglevel=warn for quiet output.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-14 18:52:43 -07:00
Anders Kaseorg a659542d84 parallel: Remove run_parallel.
This was a broken abstraction that returned to its caller within
multiple forked processes on exceptions, and encouraged ignoring the
error code (as all of its callers did).

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-09-14 16:22:23 -07:00
Anders Kaseorg c21818e093 check-openapi: Disallow siblings of $ref.
The specification says “any sibling elements of a $ref are ignored”,
so their presence, although not invalid, indicates a mistake.  yamole
incorrectly merges these siblings into the referenced object, but we
should not rely on this nonstandard behavior.

https://swagger.io/docs/specification/using-ref/#sibling

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-12 11:57:13 -07:00
Tim Abbott 45cd979d46 tools: Fix webpack wrapper logic for finding config file to watch. 2020-09-12 11:31:44 -07:00
Alex Vandiver 9ea9752e0e webhooks: Rename UnexpectedWebhookEventType to UnsupportedWebhookEventType.
Any exception is an "unexpected event", which means talking about
having an "unexpected event logger" or "unexpected event exception" is
confusing.  As the error message in `exceptions.py` already explains,
this is about an _unsupported_ event type.

This also switches the path that these exceptions are written to,
accordingly.
2020-09-10 17:47:21 -07:00
Alex Vandiver 8016769613 webhooks: Move UnexpectedWebhookEventType into zerver.lib.exceptions.
8e10ab282a moved UnexpectedWebhookEventType into
`zerver.lib.exceptions`, but left the import into
`zserver.lib.webhooks.common` so that webhooks could continue to
import the exception from there.

This clutters things and adds complexity; there is no compelling
reason that the exception's source of truth should not move alongside
all other exceptions.
2020-09-10 17:47:21 -07:00
Anders Kaseorg b7b7475672 python: Use standard secrets module to generate random tokens.
There are three functional side effects:

• Correct an insignificant but mathematically offensive bias toward
repeated characters in generate_api_key introduced in commit
47b4283c4b4c70ecde4d3c8de871c90ee2506d87; its entropy is increased
from 190.52864 bits to 190.53428 bits.

• Use the base32 alphabet in confirmation.models.generate_key; its
entropy is reduced from 124.07820 bits to the documented 120 bits, but
now it uses 1 syscall instead of 24.

• Use the base32 alphabet in get_bigbluebutton_url; its entropy is
reduced from 51.69925 bits to 50 bits, but now it uses 1 syscall
instead of 10.

(The base32 alphabet is A-Z 2-7.  We could probably replace all of
these with plain secrets.token_urlsafe, since I expect most callers
can handle the full urlsafe_b64 alphabet A-Z a-z 0-9 - _ without
problems.)

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-09 15:52:57 -07:00
Dinesh 56546170cf puppeteer: Save pid of `run-dev.py --test` in var/puppeteer.
Also replaces a reference to casper in a comment related to --test.
2020-09-09 13:38:39 -04:00
Dinesh ad0f0564b1 puppeteer: Rename build_for_prod_or_casper to build_for_prod_or_puppeteer. 2020-09-09 13:38:39 -04:00
Dinesh c64888048f puppeteer: Rename CASPER_TESTS env variable to PUPPETEER_TESTS.
Also modified few comments to match with the changes.
2020-09-09 13:38:39 -04:00
Dinesh f2b3afcd56 puppeteer: Mention about docs and screeshots on failure. 2020-09-09 13:38:39 -04:00
Anders Kaseorg 3b301f522b python: Tweak some magic trailing commas to avoid Black bugs.
https://github.com/psf/black/issues/1658
https://github.com/psf/black/issues/1671

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-03 17:51:09 -07:00
Anders Kaseorg f91d287447 python: Pre-fix a few spots for better Black formatting.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-03 17:51:09 -07:00
Anders Kaseorg bb4fc3c4c7 python: Prefer --flag=option over --flag option.
For less inflation by Black.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-03 17:51:09 -07:00
Anders Kaseorg 75c59a820d python: Convert subprocess.Popen.communicate to run or check_output.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-03 17:42:35 -07:00
Anders Kaseorg 37028d5abf zulip-export: Specify --stream argument as required.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-03 16:17:14 -07:00
Anders Kaseorg a50fae89e2 python: Elide type=str from argparse arguments.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-03 16:17:14 -07:00
Anders Kaseorg fbfd4b399d python: Elide action="store" for argparse arguments.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-03 16:17:14 -07:00
Anders Kaseorg 1f2ac1962f python: Elide default=None for argparse arguments.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-03 16:17:14 -07:00
Anders Kaseorg b4597a8ca8 python: Elide default for store_{true,false} argparse arguments.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-03 16:17:14 -07:00
Anders Kaseorg 3b257d10df tools: Delete tagmessages script.
It was broken by commit aaedec1fdb which
moved it to tools/i18n without adjusting its relative path references,
and it contains a sketchy injectable os.system call that I’d like to
remove.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-03 10:35:51 -07:00
Anders Kaseorg 5a1104f1b8 update-zuliprc-api-field: Add missing shell quoting.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-03 10:34:27 -07:00
Anders Kaseorg d751e0cece puppet: Don’t install netcat.
It’s been unused since commit 0af22dad18
(#13239).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-03 10:33:47 -07:00
Steve Howell 81b6583d28 node tests: Show coverage link when we regress coverage.
In 6653e19e3a we added
the convenient line to tell folks about the coverage
report.  But if we failed coverage checks, we didn't
show the link.

Arguably we should just always show this, even if
tests fail, but that can also be potentially confusing.
2020-09-03 07:57:58 -04:00
Steve Howell dc1795a3da node tests: Find files in Python.
The code to run single files was added
in c15695e514,
and it's just kinda strange code.

We already do a lot of file logic in Python
to check for line-coverage, so it's easier
to just have all the logic in Python.

This adds a new feature--you can now specify
the actual file:

    ./tools/test-js-with-node frontend_tests/node_tests/people.js

(This is helpful if you just want to use
shell autocomplete.)

Another minor change is that if you specify
individual files, we won't sort them.  This is
important when you're trying to hunt down test
leaks.

Finally, we have a nicer message if we can't find
the file.
2020-09-03 07:57:58 -04:00
Steve Howell e9bbfbc624 node tests: Clean up nyc arguments.
nyc was added in 29f04511c0

All the stuff after "&&" was actually passed to
node, because we didn't use shell=True, so the
"nyc report" command didn't run, and the ugly
finder.js code just skipped over all the final tokens.
2020-09-03 07:57:58 -04:00
Anders Kaseorg 0682d5c0f3 provision: Convert "".format to Python 3.6 f-strings.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-02 17:27:14 -07:00
Anders Kaseorg 424b441bb1 provision: Check for old Ubuntu or Python before starting Python.
This unblocks us from being able to use Python 3.6 syntax in
provision.py and its dependencies.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-02 17:27:14 -07:00
Anders Kaseorg 691f8f8e47 test_hash_reqs: Use mock from unittest.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-02 11:26:12 -07:00
Anders Kaseorg a276eefcfe python: Rewrite dict() as {}.
Suggested by the flake8-comprehensions plugin.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-02 11:15:41 -07:00
Anders Kaseorg 1ded51aa9d python: Replace list literal concatenation with * unpacking.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-02 11:15:41 -07:00
Anders Kaseorg a5dbab8fb0 python: Remove redundant dest for argparse arguments.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-02 11:04:10 -07:00
Anders Kaseorg edaed497ed lint: Remove unused ignorelongline and lint:ignore comments.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-02 11:00:44 -07:00
Anders Kaseorg 279e4b819e js: Elide .js and .ts extensions from imports and requires.
This will be required for TypeScript.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-01 16:43:02 -07:00
Anders Kaseorg 10bd7a47c4 webpack: Move webpack.config.ts to top level.
We lost the war against top level configuration files many moons ago.
This is what developers and tools expect.  And it seems to be required
for eslint-import-resolver-webpack (there’s ostensibly a {"config":
"tools/webpack.config.ts"} option, but it doesn’t work correctly:
https://github.com/benmosher/eslint-plugin-import/issues/1861).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-01 16:43:02 -07:00
Ryan Rehman b4f315698f node tests: Create file for `stream_edit.js`.
This fixes the regression introduced in the pervious
commit to regain the 100% line coverage in `user_pill.js`
as well as `stream_pill.js`.

The new `stream_edit.js` mainly tests for:

* The stream related queries of the typeahead in `user_pill.js`
* The "Add subscribers" event handlers.
* The event handler which displays the settings for a stream.
2020-08-31 16:57:32 -07:00
Ryan Rehman c6d9a87d6f streams: Allow creating stream pills to submit Add subscriber form.
We update the pills typeahead logic to also include
stream results and pass the "stream" key in `opts`
to enable this option for the Add subscriber form.

This commit implements the feature of adding all the
subscribers of another stream in the "Add subscribers"
UI, with the help of a new "stream_pill.js` file.

We temporarily add `user_pill.js` to the EXEMPT_FILES
list as typeahead will be set up in `stream_edit.js`
file which does not have any dedicated tests file.

Work towards #15186.
2020-08-31 16:57:32 -07:00
Aman Agrawal 6a2c7327cc boostrap: Reset to v2.3.2.
We merge bootstrap-responsive.css into bootsrap.css since that is
how bootstrap distributes it from this version onwards.

bootstrap.js has a lot of changes to it which completely breaks
our typeaheads and popovers, so we will have to override these
plugins with our version of these plugins. In future versions
of bootstrap when we use npm, we can just choose not to
import them.
2020-08-31 14:01:11 -07:00
Priyank Patel c1ba648b4f dependencies: Add xvfb dependency.
The node package allow use to control xvfb apt package in puppeteer
tests. This help us create a fake display so we can run puppeteer in
headful (headless: false) mode, which is required to use the chrome
extension desktop capture API.
2020-08-30 23:38:52 -07:00
Tim Abbott 2f5f5d7749 dependencies: Remove PhantomJS from development environment. 2020-08-30 17:16:02 -07:00
Tim Abbott 992c6126a8 docs: Update many references to Casper. 2020-08-30 17:16:02 -07:00
Dinesh 2d22d88700 casper: Remove few traces of casper.
Now that all casper tests have been migrated to
puppeteer, there's no need for having casper
related things.

Removed the casperjs package and removed/replaced
casper in few places with puppeteer.

Only removed few of them which I'm confident
about. Also didn't make any changes in docs
as it would be easier to remove them while
adding puppeteer docs.
2020-08-30 17:16:02 -07:00
Alex Vandiver 1e714d8a0f lint: Catch mock asserts whose leading "assert_" is missing. 2020-08-30 15:24:01 -07:00
Tim Abbott 26deac7904 lint: Add template check for invalid translation tag.
This rule is a bit marginal, in that we've only seen this mistake
once, but it is really subtle and took a while for translators to
notice it, so seems worth linting for anyway.
2020-08-27 11:46:41 -07:00
Tim Abbott dbf3894c49 test-backend: Enable --ban-console-output and document. 2020-08-27 11:39:53 -07:00
Mohit Gupta 3390a70bcd tests: Add extra console output detection in test-backend output.
This commit adds automatic detection of extra output (other than
printed by testing library or tools) in stderr and stdout by code under
test test-backend when it is run with flag --ban-console-output.
It also prints the test that produced the extra console output.

Fixes: #1587.
2020-08-27 11:39:53 -07:00
Aman 88821b960a vagrant: Add parallels as a provider.
There are file sharing issues with the macOS 10.15.6 and
vagrant. var/remote_cache_prefix was an empty file when using
VirtualBox and Docker on macOS.
Using parallels as a provider for vagrant fixes the issue.

Use --watch-poll which makes webpack to recompile
automatically on file changes, since inotify is not
working here too.
2020-08-14 16:03:51 -07:00
Steve Howell 196df919f2 minor: Improve error message for check-node-fixtures.
This is helpful now, but it will also reduce confusion
when we extend this tool to also look at openapi schemas.
2020-08-14 10:40:29 -04:00
Anders Kaseorg d0f4af5f8c python: Catch JSONDecodeError instead of ValueError when decoding JSON.
These weren’t wrong since orjson.JSONDecodeError subclasses
json.JSONDecodeError which subclasses ValueError, but the more
specific ones express the intention more clearly.

(ujson raised ValueError directly, as did json in Python 2.)

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-12 11:59:59 -07:00
Anders Kaseorg 42aa916967 sanity_check: Test for the virtualenv by its path.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-11 12:30:26 -07:00
Anders Kaseorg 61d0417e75 python: Replace ujson with orjson.
Fixes #6507.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-11 10:55:12 -07:00
Anders Kaseorg 768f9f93cd docs: Capitalize Markdown consistently.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-11 10:23:06 -07:00
Alex Vandiver 39368cad3a tornado: Extract functions called from django into one module.
This makes clearer the separation of concerns.
2020-08-10 16:55:56 -07:00
Anders Kaseorg 18dcb86de5 Revert "provision: Fix missing <sasl/sasl.h> headers during provision."
This reverts commit cc9dc07b07.  It was
merged with no review and has various issues (see #16086).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-10 12:09:37 -07:00
Aman cc9dc07b07 provision: Fix missing <sasl/sasl.h> headers during provision.
This fixes the error of missing sasl.h when compiling python-ldap
from source during provision.
2020-08-09 23:21:13 -07:00
Anders Kaseorg f63d132276 js: Use unminified KaTeX (and minifiy it with everything else).
Since our Webpack config passes pre-minified JS files to
script-loader, they can’t be used as modules.  Use the normal
unminified version, letting Webpack minify it and give us source maps.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 11:27:44 -07:00
Anders Kaseorg dbdf67301b memcached: Switch from pylibmc to python-binary-memcached.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-06 12:51:14 -07:00
Tim Abbott 907b16aa23 node tests: Disable coverage for recent_senders.
This was lost in 5142bdd317.
2020-08-06 12:29:27 -07:00
Anders Kaseorg 9f2cd66a4d lint: Migrate all custom CSS rules to stylelint.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-06 11:30:07 -07:00
Anders Kaseorg a1e90917d6 lint: Remove custom whitespace rules obsoleted by Prettier.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-04 16:54:03 -07:00
Anders Kaseorg c6ed5c81de styles: Format CSS with Prettier.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-04 16:34:19 -07:00
Anders Kaseorg 6c0afbb68d stylelint: Move .stylelintrc to stylelint.config.js.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-04 16:32:04 -07:00
Priyank Patel 9504d403b3 github-actions: Fix incorrect hash key in production install job.
The hash keys were missing hash for package.json and yarn.lock
because they were not present since we don't do a full checkout
in this job. We fix this by sending over those files and generating
hashes from them.

I usally verify these cache keys by clicking the Restore <cache>
step dropdown menu and then clicking the Run ... dropdown menu again
to see the generated hash.
2020-08-03 12:37:20 -07:00
Anders Kaseorg 6ec808b8df js: Add "use strict" directive to CommonJS files.
ES and TypeScript modules are strict by default and don’t need this
directive.  ESLint will remind us to add it to new CommonJS files and
remove it from ES and TypeScript modules.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-31 22:09:46 -07:00
YashRE42 b4891a5841 navbar: Rename tab_bar to message_view_header.
Prior to commit eb4a2b9d4e the center
area of the navbar was based on a structure that appended crumbs or
"tabs" as <li>s, forming a tab_bar and a tab_list.

However, in eb4a2b9d4e we apply a new
style and structure to the navbar which lets go of the convention of
tabs. Hence, we'd like to purge the tab_bar and tab_list labels from
our code base.

We purged tab_list in 1267caf5009118875f47fdafe312880af08024e1.

This commit purges tab_bar, it includes:
- A blanket search and replace of tab_bar with message_view_header.
- Splitting a single line comment in
  tab_bar.js / message_view_header.js.
- The renaming of tab_bar.js to message_view_header.js.
- The renaming of tab_bar.hbs to message_view_header.hbs.
- A blanket search and replace of tab_data with
  message_view_header_data.
- Replacing the single occurrence of tabbar with message_view_header
  (it was within a comment.)
2020-07-30 16:23:13 -07:00
Anders Kaseorg 9d768b79c5 semgrep: Use upstream options for silencing extra messages.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-30 15:12:22 -07:00
Anders Kaseorg d35b5027bb semgrep: Upgrade semgrep to 0.17.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-30 15:12:22 -07:00
Priyank Patel dda7e938f4 puppeteer: Fix flake with compose box input.
There were a lots of flakes in CI recently because typeahead didn't
appear when Enter was pressed and real emails are not accepted as
valid inputs. To fix this we wait for typeahead to appear and then
click that instead of Enter. We also use delay option to type the
email (100ms delay between keypresses) since without we'd also get
flakes.

Re-enable puppeteer test in CI after this fix too.
2020-07-30 11:32:32 -07:00
Anders Kaseorg 7ee974729b docs: Replace Digital Ocean with DigitalOcean.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-30 11:30:43 -07:00
Anders Kaseorg 8cc34d168e webpack: Remove stale reference to webpack-helpers.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-29 16:46:08 -07:00
Anders Kaseorg 9e215093ca debug-require: Remove ES6 syntax.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-29 15:28:48 -07:00
Gittenburg 6081e0228b emoji: Make 😛 have open eyes.
😛 should be the most general version, which is the one
with open eyes. Other apps do the same and it also means that :P, which
is converted to 😛 is rendered like the emoticon.

Fixes #15970.
2020-07-29 11:18:43 -07:00
Anders Kaseorg d5d6e43453 debug-require: Throw errors for unknown and unloaded modules.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-29 11:17:24 -07:00
Anders Kaseorg 11b0303d55 webpack: Reference webpack-dev-server types with triple-slash directive.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-29 11:17:24 -07:00
Anders Kaseorg d5176f4724 webpack: Inline the rest of webpack-helpers.
There’s no reason to separate this for one loader declaration.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-29 11:17:24 -07:00
Anders Kaseorg b9fd2a2b37 webpack: Inline getExposeLoaders abstraction.
We only have two of these and don’t intend to add more.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-29 11:17:24 -07:00
Anders Kaseorg 767ad1d90e debug: Remove debug from globals.
This is used rarely enough that it’s easier to document how to use it
as a non-global than to document the horrifying things that might go
wrong as a global.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-29 11:17:24 -07:00
Anders Kaseorg 226598edc4 js: Use Handlebars as a module.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-29 11:17:24 -07:00
Steve Howell 46e80cdd3a check-node-fixtures: Enforce sorting of events.
We want lib/events.js to be somewhat of an
executable piece of documentation, and it's
really not that hard to keep it sorted going
forward.
2020-07-29 08:37:18 -04:00
Tim Abbott cee7e8e89d ci: Disable puppeteer tests. 2020-07-28 17:12:49 -07:00
Anders Kaseorg 0e0baca41e typescript: Enable strictNullChecks.
Let’s do this right from the beginning instead of making a mess to
clean up later.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-28 11:01:35 -07:00
Anders Kaseorg 584d6bfa41 stats: Use Plotly.js as a module.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-28 10:54:35 -07:00
Anders Kaseorg eb09c137cf click_handlers: Use WinChan as a module.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-28 10:54:35 -07:00
Anders Kaseorg 114cc1ec25 settings_profile_fields: Use SortableJS as a module.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-28 10:54:35 -07:00
Anders Kaseorg 1d59fc272c js: Use Marked.js as a module.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-28 10:54:35 -07:00
Anders Kaseorg bcab7efb37 js: Use XDate as a module.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-28 10:54:35 -07:00
Anders Kaseorg c66931d4cd js: Use clipboard.js as a module.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-28 10:54:35 -07:00
Anders Kaseorg f0021df3d4 people: Use blueimp-md5 as a module.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-28 10:54:35 -07:00
Anders Kaseorg d2520cd7e0 js: Replace underscore with lodash and remove it from globals.
Tweaked by tabbott to bump PROVISION_VERSION.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-26 16:12:06 -07:00
Steve Howell da8691d7d7 ci: Add new check-node-fixtures tool to CI.
This tool only takes a couple seconds to run,
so it's pretty low risk to add to CI.
2020-07-26 16:07:40 -07:00
Steve Howell f9808d0c15 tools: Add tools/check-node-fixtures.
This makes sure out fixture data for node tests
is realistic, according to the schemas in
zerver/lib/event_schema.py.

Note that we are still in the process of extracting
schemas from test_events.py -> event_schema.py,
so the checks here are somewhat incomplete as of
now.

One nice thing is that the program will tell us
what checkers are missing, so this can motivate
us to move more checkers to event_schema.py.

I considered just making this happen as part of
tools/test-js-with-node, but it's convenient to
run by itself.  Also, it currently requires
Django (although we could fix that), which makes
it just expensive enough that I wouldn't want
to always run it before the node tests.
2020-07-26 16:07:33 -07:00
Priyank Patel 51199878ed tools/ci: Fix KeyError when running tools/ci/generate-dockerfiles.
This KeyError occurs because we remove extra_packages from
images.yml because it was no longer needed in a486872a8e.
2020-07-26 16:03:35 -07:00
Tim Abbott ec9c272165 help: Document process for changing organization URL. 2020-07-24 16:12:56 -07:00
Alex Vandiver a2fc823c3f provisioning: Use AWS CLI to automate provisioning
The previous steps for standing up a new host were somewhat manual.
This further scripts the process, by using the AWS CLI to start the
instance, and pass it a "user data" script to provision itself upon
boot.  This results in a hands-off provisioning process which
completes in 5min.

Additional settings are required for `~/.zulip-install-server.conf`.
It is not suited for all roles, as it assumes one instance type and
security group value.  Additionally, not all of the post-provision
process is currently automated -- Nagios SSH key verification, for
instance, is still a manual step.  There are also additional steps for
database or frontend servers.  Regardless, this is a move toward
automated provisioning.
2020-07-24 12:40:14 -07:00
Vishnu KS 3ec64b6092 team: Include users without an associated GitHub profile.
Including anon=1 in API requests will retrieve all contributors
of the repo. If there is no asscoiated GitHub account present for
the commits then the email and name of the author mentioned in
commit messages is returned.
2020-07-24 10:51:47 -07:00
Vishnu KS ecd0530578 tools: Rename server to zulip in fetch-contributor-data.
This was missed out in 2967294398
2020-07-24 10:51:47 -07:00
Priyank Patel 6a4792e844 tools/ci: Adjust ZULIP_PATH so it works for both CIs.
Until, we remove the circleci.
2020-07-24 10:36:55 -07:00
Priyank Patel 75ab26771b ci: Store production built assets to /tmp/production-build.
Previously, we copied them to /tmp and from there we specified those
assets we copied in circleci config in presist_to_workspace step.
Copying it to a directory allows us to get rid of list in circleci
config and GitHub Actions's upload artifact (their version of
presist to workspace) doesn't allow us to specify indivivual files
so only is this cleaner but required.
2020-07-24 10:36:55 -07:00
Anders Kaseorg e3b3df328d eslint: Replace sort-imports with import/order.
import/order sorts require() calls as well as import statements.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-24 09:42:56 -07:00
Anders Kaseorg 2e029eb58b lint: Use Prettier for JSON files.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-24 09:42:56 -07:00
Anders Kaseorg 40be4df57a lint: Format JSON files with Prettier.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-24 09:42:56 -07:00
Anders Kaseorg ca42fc2e21 casper_tests: Log in as Desdemona, not Iago.
As of commit 87e72ac8e2 (#15267), we
need to be an owner for some of the tested functionality, not just an
administrator.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-23 10:26:27 -07:00
Vishnu KS 9e0ff58a6d team: Rename contrib to contributors in page_params. 2020-07-23 10:22:28 -07:00
Vishnu KS 95b1a7c8d1 teams: Rename contribs_list to contributor_username_to_data. 2020-07-23 10:22:28 -07:00
Vishnu KS 2190dbd4b0 team: Map repo name to tab name in frontend.
fetch-contributor-data's job is to fetch the data. How the data
is presented in frontend is something it don't have to know about.
2020-07-23 10:22:28 -07:00
Vishnu KS fe9b700fab team: Use a better API for getting contributor data. 2020-07-23 10:22:28 -07:00
Gittenburg 45e19dd6b9 emoji: Rename :slight_smile: to 😄.
Zulip converts :) to the 1F642 Unicode emoji and promotes the same emoji
in the popular section of the emoji picker.

Previously Zulip has labeled 1F642 as "slight smile". While that name
conforms to the Unicode standard (which describes the code point as
SLIGHTLY SMILING FACE), it didn't match our use case of the emoji.

If a user types :) or selects the first smile in the emoji picker they
probably mean to express a regular "smile" and not a "slight smile",
which raises the question why they are only smiling slightly.

This commit relabels 1F642 as 😄 and our previous 😄 263A as
:smiling_face:. Note that 263A looks different in our three supported
emoji sets, so it is not suited to be our "default smile".

This change does not require a migration since our emoji system stores
both unicode points and names and handles name changes transparently.
2020-07-21 16:49:54 -07:00
Anders Kaseorg 308883302d lint: Make prettier quieter when fixing.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-21 16:31:18 -07:00
Tim Abbott b350c6ffb7 lint: Add DevAuthBackend to capitalized strings. 2020-07-21 13:23:58 -07:00
Anders Kaseorg 96dcc0ce6e js: Use ES6 object literal shorthand syntax.
Generated by ESLint.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-21 12:42:22 -07:00
Anders Kaseorg aa16208fd8 dependencies: Upgrade JavaScript dependencies.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-20 10:56:31 -07:00
Anders Kaseorg ed183fb95f lint: Use Prettier for JavaScript files.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-17 14:31:25 -07:00
Anders Kaseorg b65d2e063d js: Reformat with Prettier.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-17 14:31:25 -07:00
Anders Kaseorg 8046b6477a js: Remove extra consecutive spaces.
Prettier would do this anyway, but it’s separated out for a more
reviewable diff.  Generated by ESLint.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-17 14:31:25 -07:00
Anders Kaseorg 883e2fd325 js: Remove inner spacing from object literals.
We’re configuring Prettier with bracketSpacing: false.  Generated by
ESLint.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-17 14:31:25 -07:00
Anders Kaseorg f3726db89a js: Normalize strings to double quotes.
Prettier would do this anyway, but it’s separated out for a more
reviewable diff.  Generated by ESLint.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-17 14:31:24 -07:00
Anders Kaseorg facd860a6c lint: Exclude i18n string concatenation exposed by Prettier.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-17 14:30:52 -07:00
Steve Howell c44500175d database: Remove short_name from UserProfile.
A few major themes here:

    - We remove short_name from UserProfile
      and add the appropriate migration.

    - We remove short_name from various
      cache-related lists of fields.

    - We allow import tools to continue to
      write short_name to their export files,
      and then we simply ignore the field
      at import time.

    - We change functions like do_create_user,
      create_user_profile, etc.

    - We keep short_name in the /json/bots
      API.  (It actually gets turned into
      an email.)

    - We don't modify our LDAP code much
      here.
2020-07-17 11:15:15 -07:00
Steve Howell b375581f58 api: Remove (sender_)short_name from message APIs. 2020-07-17 11:15:14 -07:00
Anders Kaseorg 56e8cca025 lint: Add Prettier as a linter for YAML files.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-14 16:25:31 -07:00
Anders Kaseorg 2794bc1ef4 lint: Reformat YAML files with Prettier.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-14 16:25:31 -07:00
Anders Kaseorg 8927d2f240 lint: Restore pycodestyle wrapper’s check for empty list of files.
Commit ab647abad3 (#15779) accidentally
removed this.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-14 13:54:58 -07:00
Anders Kaseorg f364414cb9 install-semgrep: Upgrade semgrep to 0.14.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-14 00:42:07 -07:00
Anders Kaseorg ab647abad3 lint: Use standard setup.cfg configuration for pycodestyle.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-14 00:41:20 -07:00
Anders Kaseorg 6189e4d0c1 python: Convert more percent formatting to "".format.
Semgrep has gotten a little more clever at applying the percent
formatting rule.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-13 13:16:38 -07:00
Aman Agrawal 685ec2a098 hash_reqs: Include python version when generating hash.
Fixes #12868.
We now also include python version in the format
'major.minor.patchlevel', when generating hash for a
requirement file. This was necessary since packages tend to
break on different versions of python, so it is important to
track the version on which the venv was setup.

WARN: This commit will force all zulip venvs to be recreated.
2020-07-13 13:06:15 -07:00
Mohit Gupta 308643cace tests: Rename test_narrow.py to test_message_fetch.py.
This commit renames test_narrow.py and also all its references to
test_message_fetch.py.
2020-07-07 17:14:10 -07:00
arpit551 4f6cd6577c http_headers: Substitute Nginx version based on OS.
success-http-headers-bionic.txt and success-http-headers-focal.txt
differ only in the nginx version so this substitution will allow
us to have single file for both of them. Also this change helps
to avoid CI failure if Nginx version is updated in the OS.
2020-07-07 11:20:05 -07:00
Alex Vandiver d49e73e087 test-install: Do not install rabbitmq in base install.
The installer does not adjust the node name if the rabbitmq already
exists, and the default node name bakes in the
`zulip-install-bionic-base` hostname.  As such, the resulting LXC
image does not properly start rabbitmq.

Remove rabbitmq, allowing the installer to install and configure it
with a nodename of `zulip@localhost`.  This also lets the installed
image be successfully copied and booted under a new hostname without
breaking rabbitmq.
2020-07-06 18:53:38 -07:00
Anders Kaseorg 5693f63bf8 settings: Convert variable type annotations to Python 3.6 style.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-06 18:44:58 -07:00
Anders Kaseorg ebf7f4d0f6 zthumbor: Rename thumbor.conf to thumbor_settings.py.
So we can apply all our lint checks to it.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-06 18:44:58 -07:00
Anders Kaseorg 9900298315 zthumbor: Remove Python 2 residue.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-06 18:44:58 -07:00
Priyank Patel 29dd0c485a ci: Move upgrading git for bionic to docker image.
Just reduces the clutter in configuration file for GitHub Actions.
2020-07-06 18:09:15 -07:00
Priyank Patel c908278237 ci: Install moreutils in the docker image. 2020-07-06 18:09:15 -07:00
Priyank Patel e834439d82 provision: Do service <service-name> start instead of restart.
Doing service memcached start instead of restart fixed an issue on
focal build in GitHub actions, where it exits with code 1 when it
is done twice.It is done first in Install Dependencies step and then
again in last step where we call tools/ci/setup-backend again which
runs provision.

Furthermore, I don't belive there is a technical reason we use
restart over start; rather I think it was just a random choice with
the intend to just start the services in CI. I traced the code back
to commit 1f2f497cab if it helps.
Looking at the source code of memcached, the step that's failing is:
  start-stop-daemon --stop --quiet --oknodo --pidfile $PIDFILE
which is equivilent to: service memcached stop, we can rule out the
service memcache start since it works. Ideally, we do figure out and
solve the issue of why memcached fails when executing service
memcached stop but I am not equipped with debugging it. And this
workaround seems reasonable rather than a "hacky" solution.

For the relevant code in memcached see:
https://github.com/memcached/memcached/blob/master/scripts/memcached-init.

Finally, the change to the rest of services is for consistency.
2020-07-06 18:09:15 -07:00
arpit551 01f12b9fc2 audit_log: Log acting_user in user creation and user activation. 2020-07-06 17:32:09 -07:00
arpit551 653928bdfe audit_log: Log acting_user in do_change_avatar_fields. 2020-07-06 17:24:18 -07:00
Aman Agrawal 261bc043c9 thumbor: Replace comment type annotations with normal annotations.
Run custom lint checks on thumbor files.
2020-07-06 16:09:53 -07:00
Aman Agrawal a486872a8e requirements: Upgrade Thumbor to 7.0.0a5 on Python 3.
Co-authored-by: Anders Kaseorg <anders@zulip.com>
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-06 16:09:53 -07:00
Anders Kaseorg 8a03f8a070 html_branches: Fix strict_optional errors.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-06 11:25:48 -07:00
Anders Kaseorg e014ea966a eslint: Enable comma-dangle for functions.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-03 16:55:51 -07:00
arpit551 834abefdb9 http_headers: Update Nginx version in success-http-headers-focal. 2020-07-02 22:42:22 +05:30
Anders Kaseorg 8a54dc43ce lint: Upgrade zulint and remove run-isort wrapper.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-01 12:53:21 -07:00
Anders Kaseorg 92d0290dc5 tests: Remove slow tests detection.
According to @showell:

> All the slow decorators can die. That was a failed experiment of
> mine from 2014 days.  I have meaning to kill them for a couple years
> now.  I wrote this with the best of intentions, but I believe it's
> now just cruft.  We never made a "fast" mode, for one. And we kept
> writing more and more slow tests, haha.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-30 23:34:53 -07:00
Anders Kaseorg fa89d1b266 generate-self-signed-cert: Correct subjectAltName for an IP address.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-29 22:19:47 -07:00
Alex Vandiver 2540b678b0 ci: Test upgrading PostgreSQL 10 -> 12. 2020-06-29 17:18:47 -07:00
Alex Vandiver f621e1b7e4 ci: Split out installation testing. 2020-06-29 17:18:47 -07:00
Alex Vandiver 36335ecd39 ci: Simplify tool-copying in production-build. 2020-06-29 17:18:47 -07:00
Priyank Patel 92f98292c1 tools/ci: Create a github user instead of a circleci user.
While we are making this change, we create a ARG USERNAME variable
that behaves like ENV but just for the lifetime of the Dockerfile.
2020-06-29 16:31:43 -07:00
Priyank Patel a2c2c6ceff tools: Move CI docker images files into tools/ci. 2020-06-29 16:31:43 -07:00
Priyank Patel b1c0c13008 provision: Include check for GitHub Action along with CircleCI. 2020-06-29 16:31:43 -07:00
Gittenburg a09ee7818c emoji: Convert :| to :neutral:.
Previously we converted it to 😑 (which however has closed
eyes). :neutral: has open eyes and therefore more closely resembles :|
2020-06-29 16:31:20 -07:00
Gittenburg 0ad67a2c5f emoji: Add 6 more emoticon conversions.
While there are many more these are well established.
2020-06-29 16:31:20 -07:00
Mohit Gupta c16c713060 refactor: Rename bugdown to markdown in misc. files.
This commit is part of series of commits aimed at renaming bugdown to
markdown.
2020-06-29 15:10:39 -07:00
Mohit Gupta 4224ac1b61 refactor: Rename remaining bugdown words to markdown in test_markdown.py.
Rename rest of function names, classes and comments containing bugdoown
to markdown in test_markdown.py. Also change the refactored classes and
functions occurences in other files.
This commit is part of series of commits aimed at renaming bugdown to
markdown.
2020-06-26 17:23:15 -07:00
Mohit Gupta 0578a918e6 refactor: Rename test_bugdown.py to test_markdown.py.
Rename the file and all the refrences to file and module test_bugdown.py
to test_markdown.py.
This commit is part of series of commit that renames bugdown to markdown.
2020-06-26 17:08:37 -07:00
Mohit Gupta 3f5fc13491 refactor: Rename zerver.lib.bugdown to zerver.lib.markdown .
This commit is first of few commita which aim to change all the
bugdown references to markdown. This commits rename the files,
file path mentions and change the imports.
Variables and other references to bugdown will be renamed in susequent
commits.
2020-06-26 17:08:37 -07:00
Vishnu KS ce6203906f i18n: Mark notification bot message in notify_new_user for translation. 2020-06-26 14:57:16 -07:00
Anders Kaseorg 4a88e2a732 semgrep: Ban eval.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-26 12:35:40 -07:00
Anders Kaseorg ebb2efa664 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-25 13:14:37 -07:00
Anders Kaseorg 123d527c9e check-issue-labels: Fix configparser import.
Commit ccf2792c1c (#7296) migrated this
incorrectly.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-24 19:26:02 -07:00
Alex Vandiver e0b725be63 test-install: Wait for network in the lxc container.
Ubuntu 20.04 "focal" comes up to runlevel 5 several seconds before it
is able to successfully resolve hosts, causing `prepare-base` to fail
while fetching from the apt repositories.

Add an additional check to verify that outbound networking is running
before returning from `lxc-wait`.
2020-06-24 16:07:20 -07:00
Alex Vandiver 8499b2c0dc test-install: Add support for focal. 2020-06-24 13:03:13 -07:00
Alex Vandiver 8719fba3c4 test-install: Stop installing postgres-10.
As in the previous commit, we can no longer pre-install the wrong
version of postgres.  Unfortunately, this leaves it out of the base
image and thus makes testing installs longer.
2020-06-24 13:03:13 -07:00
arpit551 e19aef569a ci: Update Circle CI images.
Removed postgresql and python 2 packages when creating docker images
for Circle CI. Updated the images.
2020-06-24 11:19:47 -07:00
Tim Abbott b7b173d2ae realm: Fix type-checking for message_retention_days.
The best way to handle this is to have the potentially string-typed
value be a separate variable.
2020-06-24 11:01:34 -07:00
Anders Kaseorg d6655689f5 validator: Remove unused type_structure system.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-23 16:30:17 -07:00
Anders Kaseorg 48719cb8c4 message_send: Avoid unchecked cast.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-23 13:30:09 -07:00
Anders Kaseorg 3ffed617a2 mypy: Type simple generators as Iterator, not Iterable.
A generator that yields values without receiving or returning them is
an Iterator.  Although every Iterator happens to be iterable, Iterable
is a confusing annotation for generators because a generator is only
iterable once.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-23 11:29:54 -07:00
Felix a389c7390d calls: Add Big Blue Button as a Video Call Provider.
Big Blue Button needs an API secret so communication to creating a
room has to be done server side.

Fixes #14763.
2020-06-22 16:19:07 -07:00
Priyank Patel 89619767cd test_server: Double the timeout amount to avoid flakes.
These timeout flakes, when running puppeteer test, have been fairly
frequent in CI, so increase the timeout.
2020-06-22 12:30:32 -07:00
Aman Agrawal a052d24231 recent_topics: Support arrow key navigation.
Add arrow key navigation support for recent topics.

Simple jquery is used to allow navigation for filter buttons,
a grid system is used for navigation inside table.
2020-06-20 22:50:47 -07:00
Steve Howell 22ead64d54 tests: Extract USING_TYPE_STRUCTURE. 2020-06-20 05:22:35 -04:00
Anders Kaseorg 3916ea23a9 python: Combine some split import groups.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-18 15:54:11 -07:00
Clara Dantas e87df351f7 renumber-migrations: Skip migration 0261.
Because of a security release that required a migration, there
are two migrations numbered 0261. To avoid breaking existing
installs renumbering the migrations, we skipped migration 0261
when running tools/renumber-migrations.
2020-06-18 15:42:57 -07:00
Dinesh 0445311430 auth: Make apple log in and sign up buttons consistent with others. 2020-06-18 13:06:10 -07:00
Tim Abbott ba1f2dfdfa pointer: Merge pointer.js into message_scroll.js.
Ths remaining code here is naturally message scrolling logic, and fits
well into the existing file for handling that.
2020-06-18 12:55:59 -07:00
Dinesh 3d97050e75 puppeteer: Improve --interactive option for test-js-with-puppeteer.
Previously, --interactive used to run tests from the start on a
repeated run triggered when tests failed and we decide to trigger
a re-run by pressing Enter key. Rerunning passed tests is of no interest.
It also used to run all tests in a loop even if all pass.

This commit fixes those both issues i.e it runs again from the
test that failed on pressing Enter and exits if all tests pass.
2020-06-17 15:24:14 -07:00
Tim Abbott 4fff858aa2 templates: Fix missing quoting of attributes in HTML templates.
This fixes a bundle of issues where we were missing "" around
attributes coming from variables.  In most cases, the variables were
integers or fixed constants from the Zulip codebase (E.g. the name of
an installed integration), but in at least one case it was
user-provided data that could potentially have security impact.
2020-06-16 23:35:39 -07:00
Sara Gulotta 1cb040647b markdown: Add support for spoilers.
This adds support for a "spoiler" syntax in Zulip's markdown, which
can be used to hide content that one doesn't want to be immediately
visible without a click.

We use our own spoiler block syntax inspired by Zulip's existing quote
and math block markdown extensions, rather than requiring a token on
every line, as is present in some other markdown spoiler
implementations.

Fixes #5802.

Co-authored-by: Dylan Nugent <dylnuge@gmail.com>
2020-06-16 16:14:10 -07:00
Aman Agrawal 6950d8d769 provision: Don't run migrations on `zulip` db in CircleCI.
The automated tests running in CircleCI don't actually use the `zulip`
db, so we can skip running migrations on it in some CircleCI shards to
save time.

NOTE: This only effects build jobs that run provision, except the
`production-build` job where we skip building the dbs altogether.
Migrations still run on `focal-backend` build job to ensure
we are testing all our development setup code.
2020-06-16 15:37:32 +05:30
Anders Kaseorg fa2496c229 terminate-psql-sessions: Rely on the caller to set PGHOST, PGUSER.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-15 21:37:14 -07:00
Alex Vandiver 7fe1780088 provision: Install `crudini` for zulip.conf shell scripting.
A few tools already use this, and it is part of server installation;
install it in development environments as well.
2020-06-15 17:58:39 -07:00
Anders Kaseorg cf6981eef0 lint: Remove other rules about percent formatting.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-15 16:24:46 -07:00
Anders Kaseorg f364d06fb5 python: Convert percent formatting to .format for translated strings.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-15 16:24:46 -07:00
Anders Kaseorg 769e198ab0 lint: Prohibit percent formatting on literal format strings.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-14 23:27:22 -07:00
Anders Kaseorg 5dc9b55c43 python: Manually convert more percent-formatting to f-strings.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-14 23:27:22 -07:00
Anders Kaseorg 3461db7ef5 python: Convert percent formatting to "".format in certain files.
These files can’t use f-strings yet because they need to run in Python
2 or Python 3.5.

Generated by pyupgrade.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-14 23:27:22 -07:00
Anders Kaseorg 74c17bf94a python: Convert more percent formatting to Python 3.6 f-strings.
Generated by pyupgrade --py36-plus.

Now including %d, %i, %u, and multi-line strings.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-14 23:27:22 -07:00
Anders Kaseorg 1ed2d9b4a0 logging: Use logging.exception and exc_info for unexpected exceptions.
logging.exception() and logging.debug(exc_info=True),
etc. automatically include a traceback.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-14 23:27:22 -07:00
Anders Kaseorg 674158b817 migrations: Escape more pedantically in pgroonga.0003_v2_api_upgrade.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-13 21:50:37 -07:00
Anders Kaseorg 0cc897d08d migrations: Escape more pedantically in pgroonga.0001_enable.
The psycopg2.SQL API unfortunately doesn’t work with
django.db.migrations.RunSQL, so we need to take a detour into
PL/pgSQL for EXECUTE and format.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-13 21:50:37 -07:00
Anders Kaseorg 80b991fc44 production-helper: Clarify log output on check_rabbitmq_consumers failure.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-13 16:49:32 -07:00
Anders Kaseorg 57a80856a5 python: Convert more "".format to Python 3.6 f-strings.
Generated by pyupgrade --py36-plus --keep-percent-format.

Now including %d, %i, %u, and multi-line strings.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-13 15:39:00 -07:00
Steve Howell 9d9342c2a5 dead code: Remove integration_bot_widget code.
This was work in progress from summer 2017
that never got completed.

See 534aa0b105
2020-06-13 15:35:39 -07:00
Anders Kaseorg 0d6c771baf python: Guard against default value mutation with read-only types.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-13 15:31:27 -07:00
Anders Kaseorg 91a86c24f5 python: Replace None defaults with empty collections where appropriate.
Use read-only types (List ↦ Sequence, Dict ↦ Mapping, Set ↦
AbstractSet) to guard against accidental mutation of the default
value.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-13 15:31:27 -07:00
Anders Kaseorg 69c0959f34 python: Fix misuse of Optional types for optional parameters.
There seems to have been a confusion between two different uses of the
word “optional”:

• An optional parameter may be omitted and replaced with a default
  value.
• An Optional type has None as a possible value.

Sometimes an optional parameter has a default value of None, or None
is otherwise a meaningful value to provide, in which case it makes
sense for the optional parameter to have an Optional type.  But in
other cases, optional parameters should not have Optional type.  Fix
them.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-13 15:31:27 -07:00
Dinesh 7229088b6c tools: Fix message-screenshot.js failing to take screenshot.
To take screenshots, we log in as Desdemonia who doesn't seem
to have permissions to the streams where we capture screenshots.
This commit fixes it by logging in as Iago.

Also added a waitForNavigation until DOMContentLoaded to
avoid flakes that could occur because of us going to another URL
before Iago gets logged in.
2020-06-13 07:20:32 -04:00
Anders Kaseorg 59e3c1f5e4 install-semgrep: Upgrade semgrep to 0.10.1.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-11 18:23:17 -07:00
Anders Kaseorg 48e55875fe lint: Run isort as a linter.
With a change to increase the line length to 100 characters by
tabbott.

Fixes #2665.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-11 16:47:07 -07:00
Anders Kaseorg 365fe0b3d5 python: Sort imports with isort.
Fixes #2665.

Regenerated by tabbott with `lint --fix` after a rebase and change in
parameters.

Note from tabbott: In a few cases, this converts technical debt in the
form of unsorted imports into different technical debt in the form of
our largest files having very long, ugly import sequences at the
start.  I expect this change will increase pressure for us to split
those files, which isn't a bad thing.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-11 16:45:32 -07:00
Anders Kaseorg 69730a78cc python: Use trailing commas consistently.
Automatically generated by the following script, based on the output
of lint with flake8-comma:

import re
import sys

last_filename = None
last_row = None
lines = []

for msg in sys.stdin:
    m = re.match(
        r"\x1b\[35mflake8    \|\x1b\[0m \x1b\[1;31m(.+):(\d+):(\d+): (\w+)", msg
    )
    if m:
        filename, row_str, col_str, err = m.groups()
        row, col = int(row_str), int(col_str)

        if filename == last_filename:
            assert last_row != row
        else:
            if last_filename is not None:
                with open(last_filename, "w") as f:
                    f.writelines(lines)

            with open(filename) as f:
                lines = f.readlines()
            last_filename = filename
        last_row = row

        line = lines[row - 1]
        if err in ["C812", "C815"]:
            lines[row - 1] = line[: col - 1] + "," + line[col - 1 :]
        elif err in ["C819"]:
            assert line[col - 2] == ","
            lines[row - 1] = line[: col - 2] + line[col - 1 :].lstrip(" ")

if last_filename is not None:
    with open(last_filename, "w") as f:
        f.writelines(lines)

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-06-11 16:04:12 -07:00
Anders Kaseorg ca4357fd64 python: Use standard NoReturn (Python ≥ 3.6).
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-11 12:56:52 -07:00
Anders Kaseorg 67e7a3631d python: Convert percent formatting to Python 3.6 f-strings.
Generated by pyupgrade --py36-plus.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-10 15:02:09 -07:00
Anders Kaseorg 6480deaf27 python: Convert more "".format to Python 3.6 f-strings.
Generated by pyupgrade --py36-plus --keep-percent-format, with more
restrictions patched out.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-10 14:48:09 -07:00
Anders Kaseorg d3e8af4ad2 python: Replace silly uses of filter().
The test_management_commands use in particular was causing pickling
errors when the test failed, because Python 3 filter returns an
iterator, not a list.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-09 23:31:59 -07:00
Anders Kaseorg 3decbd1e23 lint: Remove settings exemption for possibly undefined star imports.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-09 22:29:50 -07:00
Anders Kaseorg 24d320f245 dev_settings: Move prod_settings_template import to configured_settings.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-09 22:20:42 -07:00
Anders Kaseorg c45962785c settings: Group {default,prod,dev}_settings as configured_settings.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-09 22:20:42 -07:00
arpit551 662f4902f8 ci: Setup production job for Focal.
Run production suites on Ubuntu Focal.
Added separate success-http-headers files for Focal and Bionic.
Also excluded them from whitespace rules in lint.

memcached 1.5.22 in Ubuntu 20.04 has a bug where it looks for its SASL
configuration at /etc/sasl2/memcached.conf/memcached.conf instead of
/etc/sasl2/memcached.conf.
We already use a workaround for this while applying puppet configurations in
99e71f3786 but for docker builds we used
do memcached hack since we can not use systemd in docker containers.
2020-06-08 21:59:57 -07:00
Anders Kaseorg 9ca5a65296 build-release-tarball: Add umask override.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-08 21:57:05 -07:00
Tim Abbott 7b8ba5ebd9 docs: Update most remaining references to zulipchat.com.
In some cases, the cleanest tweak is to replace references to the
domain with Zulip Cloud, the product.
2020-06-08 18:10:45 -07:00
Tim Abbott 95be7dcbab portico: Remove mystery hunt landing page.
This was written by Rishi for a very brief purpose a few years ago,
and it doesn't serve much purpose now other than to be a place we
update in code sweeps.
2020-06-08 18:10:45 -07:00
Anders Kaseorg 8dd83228e7 python: Convert "".format to Python 3.6 f-strings.
Generated by pyupgrade --py36-plus --keep-percent-format, but with the
NamedTuple changes reverted (see commit
ba7906a3c6, #15132).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-08 15:31:20 -07:00
Sharif Naas b0a0ae215f js: Extract message_edit_history.js. 2020-06-07 13:57:28 -07:00
Anders Kaseorg cf60493f8e install-shellcheck: Upgrade ShellCheck to 0.7.1.
ShellCheck binary releases have moved to GitHub:

https://github.com/koalaman/shellcheck/issues/1871

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-07 11:06:57 -07:00
Anders Kaseorg d49e880c09 lint: Remove custom rules duplicating eslint and pycodestyle.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-05 17:43:30 -07:00
Anders Kaseorg 5ad33c9d62 install-semgrep: Upgrade semgrep to 0.9.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-05 09:35:11 -07:00
arpit551 569731da50 provision: Removed hack for running production suites in ci.
Since now, we use different jobs for getting release-tarball
and running production suites so we don't require this hack
2020-06-04 14:26:23 -07:00
arpit551 291bce93d0 provision: Rename --production-test-suite option in provision.
Since we use this option in our docker-zulip project also
so rather than using it as a test suite option we made it
more specific i.e. --build-release-tarball-only.
2020-06-04 14:26:23 -07:00
Dinesh d429b7145e tests: Make puppeteer tests run in full suites and CircleCI. 2020-06-03 17:39:35 -07:00
Anders Kaseorg dd4d8968da custom_check: Use unconditional type imports.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-03 17:23:20 -07:00
Anders Kaseorg 789e612c70 requirements: Update Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-03 17:23:20 -07:00
Anders Kaseorg 5ab1ddbad9 test_zulint_custom_rules: Fix open mock.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-03 17:23:20 -07:00
Anders Kaseorg 7a53da7526 capitalization: Fix OAuth capitalization.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-06-03 16:39:12 -07:00
Puneeth Chaganti d2dadde381 tools: Use the correct path for bot avatar to generate screenshots. 2020-06-01 18:08:23 -07:00
Puneeth Chaganti 788203778d tools: Use the correct logo path to generate bot avatar.
Integrations can be configured with specific logo paths, which weren't
correctly being used for generating the bot avatars.
2020-06-01 18:08:23 -07:00