Commit Graph

2517 Commits

Author SHA1 Message Date
Tim Abbott f1850b99cf test-backend: Don't use monkey-patching to set formatter class.
The previous approach threw a mypy error, and was also bad code
anyway.
2017-11-21 21:55:48 -08:00
Tim Abbott 33865a3535 lint: Add exclude rules for URLs in code comments. 2017-11-21 21:53:48 -08:00
rht e55898850a Replace optparse with argparse in remaining tools.
Tweaked by tabbott to fix various bugs with the usage output.
2017-11-21 21:34:38 -08:00
Tim Abbott aa2a4632c5 tests: Lock down 100% test coverage on zerver/forms.py.
With the recent work on OurAuthenticationForm, this now has complete coverage.
2017-11-21 20:14:12 -08:00
Vishnu Ks 766511e519 actions: Mark all messages as read when user unsubscribes from stream.
This fixes a bug where, when a user is unsubscribed from a stream,
they might have unread messages on that stream leak.  While it might
seem to be a minor problem, it can cause significant problems for
computing the `unread_msgs` data structures, since it means we need to
add an extra filter for whether the user is still subscribed, either
in the backend or in the UI.

Fixes #7095.
2017-11-21 20:09:17 -08:00
Vishnu Ks a0275a6257 actions: Send new user messages to signup_notifications_stream. 2017-11-21 17:39:50 -08:00
rht bff736868e Generate custom-icon-webfont on each provision or update-prod-static.
Fixes #7354.
2017-11-20 16:36:49 -08:00
Harshit Bansal 5111aeac41 tools: Rewrite `test-locked-requirements` to be more performant.
This commit modifies `test-locked-requirements` to use some caching
so that we don't need to use the `update-locked-requirements` tool
everytime for checking the validity of locked requirements as it is
slow.

Fixes: #6969.
2017-11-20 15:35:47 -08:00
Harshit Bansal 40fe6239c5 build_emoji: Migrate to python3. 2017-11-20 15:30:58 -08:00
Harshit Bansal df4d10a3df custom_check.py: Fix the comment for ignoring long lines.
Change `#ignorelongline` to `# ignorelongline` as comments must a
have whitespace after `#`.
2017-11-20 15:29:47 -08:00
Harshit Bansal e75f0c1ee4 requirements: Rename requirements files.
This commit renames various source requirements files like `dev.txt`,
`mypy.txt` etc to `dev.in`, `mypy.in` etc and various locked requirements
files like `dev_lock.txt`, `mypy_lock.txt` etc to `dev.txt`, `mypy.txt`
etc. This will help in emphasizing to the user that *.in are actually
input to `update-locked-requirements` tool which should be run after
updating any of these.
2017-11-21 02:38:26 +05:30
Harshit Bansal 95b3f2e745 requirements: Add mypy.txt and docs.txt to locked requirements framework. 2017-11-21 02:38:26 +05:30
Tim Abbott e81b64d511 docs: Move remotedev docs to docs/development/ and fix links.
Thanks to SivagiriVisakan for pointing out the issues here; I ended up
redoing this since the doc really belongs in that directory.
2017-11-20 11:13:59 -08:00
Tim Abbott 25ddba99f6 lint: Ban general use of user_profile.save().
This often can cause minor caching problems.

Obviously, it'd be better if we had access to the AST and thus could
do this rule for UserProfile objects in general.
2017-11-20 10:57:08 -08:00
Balaji2198 c3ac9894a5 docs: Move migration-renumbering.md to subsystems/.
It belongs in that part of the documentation.  We also update the
links in docs/subsystems/schema-migrations.md.
2017-11-20 10:20:31 -08:00
Greg Price 13aa23eb40 provision: Run clean-unused-caches.
This prevents the caches in /srv from growing to fill up the disk --
e.g., on my laptop after 6 months of regular development the venv cache
was 12G and the NPM cache 5G, making them by far the largest disk hogs
on the machine.

It costs about 0.4s, apart from any time spent actually removing
things.  This is a little annoyingly slow to be adding to every
provision, and seems like it could be optimized, but I think already
worth it as is.
2017-11-18 15:04:08 -08:00
derAnfaenger 73915a9631 docs: Move integration docs to /api. 2017-11-17 11:58:54 -08:00
Tim Abbott 57c78da915 test-documentation: Build docs in parallel.
We already do this by default in tools/build-docs, but since we
migrated test-documentation to not run that directly (to disable
collapsing), we need to add the recent parallelism fix here too.

It saves about 5-10s when running this test suite for me, which is
good, but definitely leaves me feeling like there could be more
improvement.
2017-11-17 11:06:56 -08:00
Aditya Bansal dd037df2c4 thumbor: Add dependencies and virtualenv setup script.
In this commit we add new dependencies needed for running thumbor.
Also we add the script for creating the virtual environment ready
for thumbor.
Note: Thumbor will use python2 and thus have different virtualenv
dedicated to it.
Credits to @TigorC and @joshland as well for there work on this.
2017-11-16 22:38:29 -08:00
Aditya Bansal 66060c703c update-locked-requirements: Add py2 option to compile requirements.
In this commit we add a new option which could be used to specify
python version. When 'py2' is specified, future/futures are not
removed from the requirements lock file generated.
2017-11-16 22:34:14 -08:00
Tim Abbott 41a0fd97c4 build-release-tarball: Improve help/error output. 2017-11-16 22:19:58 -08:00
Tim Abbott 780d83efab build-release-tarball: Add support for --help. 2017-11-16 22:19:58 -08:00
Tim Abbott 569cb1d2fb docs: Move readme-symlink symlink to be just called readme.
This name was always a little confusing.
2017-11-16 21:43:25 -08:00
Vishnu Ks 40fca20c51 droplets: Run git clean -f after creating droplet. 2017-11-16 19:47:42 -08:00
Vishnu Ks 75601f660d droplets: Always rebase instead of merge during git pull. 2017-11-16 19:47:42 -08:00
Greg Price 0cc8d243e4 build-release-tarball: Fold away the confusing variable TMP_CHECKOUT.
Even where this is actually used for a temporary checkout, it obscures
the relationship between this and $TMPDIR -- and some of our logic
depends on that.  In other places, it isn't actually even a checkout.
In all cases, the expanded version is clearer.
2017-11-16 15:54:34 -08:00
Greg Price 3d1211edcf build-release-tarball: Cut a recently-obsoleted comment.
We no longer use .gitattributes for this purpose (since bfaf0fb3b),
so this comment isn't true anymore.
2017-11-16 15:54:34 -08:00
Greg Price 18f9e18324 build-release-tarball: Drop likely-bitrotted macOS support.
This script, and tools/update-prod-static which it relies on,
have kept getting more complex since this conditional was added
in 2013, and the places we rely on GNU features have probably
multiplied beyond `mktemp -d`.  It's unlikely this works on
macOS with BSD tools now, and it'd be hard to maintain that way
if it did; drop the pretense.
2017-11-16 15:54:34 -08:00
Greg Price 5a2f3fe9c2 build-release-tarball: Reorder a bit for clarity.
This puts our fiddling with the update-prod-static logfiles
immediately after we run update-prod-static.
2017-11-16 15:54:34 -08:00
Greg Price 93fbffebd7 build-release-tarball: Cut a redundant cleanup step.
There's no need to remove this file here -- the whole tree will be
removed a few commands later, and the `tar` command we do first, to
supplement our tarball with various generated files, is quite
selective and wouldn't look at this file anyway.
2017-11-16 15:54:34 -08:00
Greg Price 09a38ec5dc build-release-tarball: Cut a long-stranded comment.
This comment only made sense for about one week, in 2013;
the setting it points at was removed in e6ca5d265.
2017-11-16 15:54:34 -08:00
Greg Price 8d405da96c build-release-tarball: Use set +x consistently. 2017-11-16 15:54:34 -08:00
Greg Price 92e322a9d1 build-release-tarball: Add --authors-not-required.
This makes it not real annoying to run this in a development loop.
2017-11-16 15:54:34 -08:00
Tim Abbott e1f8d24166 lint: Fix tools tests.
These were broken when Travis wasn't running properly.
2017-11-16 13:59:40 -08:00
Tim Abbott 2563f93d46 test-help-documentation: Pass use_db=True.
With the new portico work we've done, the help documentation does
sorta depend on the database if you're logged in.  So it's best to
just require it for these tests.
2017-11-16 13:44:53 -08:00
Tim Abbott cb4acf2aa4 docs: Lint rst files for trailing whitespace. 2017-11-16 13:20:50 -08:00
Tim Abbott d9da8f859d lint: Check for absolute/relative link style in developer docs.
This should help avoid a pretty wide range of link bugs we've had in
the past.
2017-11-16 12:44:16 -08:00
Tim Abbott bf77ad3a6e docs: Clean a few files of too-long lines.
We also document why the rest of them are still on the list.
2017-11-16 11:49:00 -08:00
Tim Abbott 7c24e04223 lint: Fix long lines in README.md. 2017-11-16 11:38:13 -08:00
Tim Abbott 638eb7a8e4 docs: Update links to ReadTheDocs to always use https.
This is better security practice.

We also add a lint rule to enforce this for the future.
2017-11-16 10:59:24 -08:00
Tim Abbott 054952a44a docs: Update links from codebase to point to ReadTheDocs. 2017-11-16 10:53:49 -08:00
David Rosa Tamsen 7072fa5b37 docs: Reorganize developer docs to improve navigation.
This commit helps reduce clutter on the navigation sidebar.
Creates new directories and moves relevant files into them.
Modifies index.rst, symlinks, and image paths accordingly.

This commit also enables expandable/collapsible navigation items,
renames files in docs/development and docs/production,
modifies /tools/test-documentation so that it overrides a theme setting,
Also updates links to other docs, file paths in the codebase that point
to developer documents, and files that should be excluded from lint tests.

Note that this commit does not update direct links to
zulip.readthedocs.io in the codebase; those will be resolved in an
upcoming follow-up commit (it'll be easier to verify all the links
once this is merged and ReadTheDocs is updated).

Fixes #5265.
2017-11-16 09:45:08 -08:00
Tim Abbott 774380dcb8 capitalization: Allow enabled/disabled to be lower-case. 2017-11-15 17:39:09 -08:00
Tommy Ip 0a445031a9 /team: Remove responsive columns hack.
Since we are restyling the contributors table to the new fancy CSS
grid layout we no longer need this hack.
2017-11-15 17:08:08 -08:00
rht e54b9730fc Reduce python files line length limit to 110.
License: Apache-2.0
Signed-off-by: rht <rhtbot@protonmail.com>
2017-11-15 10:58:03 -08:00
rht bafba95eb7 Text-wrap the remaining long lines exceeding 110. 2017-11-15 10:58:03 -08:00
Eeshan Garg e85d0747d4 tools/test-help-documentation: Verify api/ docs. 2017-11-15 10:12:59 -08:00
derAnfaenger 5ef09bfcd4 docs: Move bot docs to /api.
With a few fixes for rendering bugs.  The rest will be in a follow-up
issue.
2017-11-15 09:58:29 -08:00
Vishnu Ks 36f29764cb emails: Add option to forward mails send in dev env to external email.
Fixes #7085.
2017-11-14 15:27:29 -08:00
Tim Abbott ac763d6eed tools: Remove obsolete build-deb tool.
This hasn't been used in ages, and isn't useful with the current
codebase.
2017-11-14 15:22:26 -08:00
rht 72ce1c6501 Use a more concise `mkdir -p` when the situation permits. 2017-11-14 09:47:09 -08:00
Steve Howell ba51078418 Simplify CSS linter and clean up CSS.
The CSS linter was pretty hard to reason about.  It was
pretty flexible about certain things, but then it would
prevent seemingly innocuous code from getting checked in.

This commit overhauls the pretty-printer to be more composable,
where every object in the AST knows how to render itself.  It
also cleans up a little bit of the pre_fluff/post_fluff logic
in the parser itself, so comments are more likely to be "attached"
to the AST node that make sense.

The linter is actually a bit more finicky about newlines, but
this is mostly a good thing, as most of the variations before
this commit were pretty arbitrary.
2017-11-13 12:43:43 -08:00
Vishnu Ks 50eb3be0c2 droplets: Remove ip_address default value from create_dns_record.
I forgot to remove the default value 172.31.1.4 from
ip_address which was used only for testing purpose.
2017-11-11 17:05:14 -08:00
rht f397e15128 tools: Text-wrap long lines exceeding 110. 2017-11-10 16:24:09 -08:00
rht 7a96b8c7ad tools/lib: Text-wrap long lines exceeding 110. 2017-11-10 16:24:09 -08:00
Vishnu Ks 07e6a4fb2d droplets: Add wildcard A records to support realm subdomains. 2017-11-10 16:13:53 -08:00
rht 5ff690d486 zulip-export: Convert to using argparse from optparse. 2017-11-10 15:54:02 -08:00
rht f689813f38 test-js-with-node: Port from optparse to argparse. 2017-11-10 15:52:43 -08:00
Cynthia Lin 0b800b0a7d icons: Create framework for custom icons and add new bot icon. 2017-11-10 11:18:42 -08:00
neiljp (Neil Pilgrim) fec59b29cd Tools/mypy: Enforce typed generics in tools/run-mypy.
This adds the "--disallow-any=generics" option to run-mypy, which no
longer permits:
- inheriting from "list"; use "List[sometype]" (or a TypeVar)
- generic types with no following square brackets specifying the type
  (even if initially 'Any')

Any (and '...' for Callable) is a lot easier to search for than an
absence of square brackets, and should improve overall typing quality.
2017-11-08 12:40:40 -08:00
derAnfaenger c9d438d149 linter: Add example lines for several python rules. 2017-11-08 11:18:37 -08:00
derAnfaenger dbc1f83d3a linter: Make string formatting custom rule more complete. 2017-11-08 11:16:18 -08:00
derAnfaenger 88794ea361 linter: Fix tests ignoring rules with `include_only`. 2017-11-08 11:16:18 -08:00
rht 9c7d5812ce refactor: Remove six.moves.urllib.parse import. 2017-11-07 10:51:44 -08:00
rht ccf2792c1c refactor: Remove six.moves.configparser import. 2017-11-07 10:51:44 -08:00
rht 6b12fe24b6 refactor: Remove six.moves.input import. 2017-11-07 10:51:44 -08:00
rht 6cce0e346e refactor: Remove six.moves.filter import. 2017-11-07 10:51:44 -08:00
rht 80a8d4f9f3 refactor: Remove six.moves.map import. 2017-11-07 10:46:42 -08:00
rht 549a26860f refactor: Remove six.moves.range import. 2017-11-07 10:46:42 -08:00
rht ec5120e807 refactor: Remove six.moves.zip import. 2017-11-07 10:46:42 -08:00
Vishnu Ks c5ff020246 droplets: Garbage collect A records before creating droplet. 2017-11-06 12:20:39 -08:00
Rishi Gupta dec4b9ed93 remote dev: Add code and instructions for creating digital ocean droplets.
Mostly copied from the zulip/zulip-gci repository, but with some changes to
wordings and code cleanup for linters.
2017-11-06 12:20:39 -08:00
rht e3eebf3be0 Remove inheritance from object. 2017-11-06 08:53:48 -08:00
neiljp (Neil Pilgrim) 737408f741 mypy: Add explicit flexible type parameters for Callable in run-dev.py. 2017-11-04 19:47:44 -07:00
neiljp (Neil Pilgrim) 9f90a3174b mypy: Add explicit Any as Dict parameters in check_line_coverage in test-js-with-node. 2017-11-04 19:47:44 -07:00
neiljp (Neil Pilgrim) abc0d5e79b mypy: Add type parameter for Set in check_help_documentation.py. 2017-11-04 19:47:44 -07:00
neiljp (Neil Pilgrim) a5d1fdf0d6 mypy: Add type parameter for IO in test-queue-worker-reload. 2017-11-04 19:47:44 -07:00
neiljp (Neil Pilgrim) dac67ebe67 mypy: Remove un-necessary comment re mypy issue #3145.
RuleList is required at global scope now, for later functions.
2017-11-04 19:22:06 -07:00
neiljp (Neil Pilgrim) 05ef052ef5 mypy: Improve typing in custom_check_file() in custom_check.py. 2017-11-04 16:18:27 -07:00
rht dcc831f767 refactor: Replace all __unicode__ method with __str__.
Close #6627.
2017-11-02 11:01:47 -07:00
Tim Abbott d27002ba21 landing-page: Replace /about with /team and /history in links. 2017-10-31 12:44:05 -07:00
neiljp (Neil Pilgrim) 966d5f3760 mypy: Specify Callable parameter/return type for run decorator in tools/diagnose. 2017-10-31 00:03:35 -07:00
Tim Abbott f01f596138 help: Merge api.js and help.js bundles to fix code blocks.
In addition to decreasing the excessive number of bundles we had, this
will set us up to fix rendering of code blocks when clicking the
sidebar links in the /api-new site.
2017-10-30 22:53:17 -07:00
Brock Whittaker e86e259cb1 /api/: Add Markdown-based /api-new documentation.
This commit allows for the /api-new/ page to rendered similarly to our
/help pages.  It's based on the old content for /api, but we're not
replacing the old content yet, to give a bit of time to restructure
things reasonably.

Tweaked by eeshangarg and tabbott.
2017-10-30 22:32:43 -07:00
Greg Price eb55a3a1ba template context: Give better names to the URLs for the API.
The "subdomain" label is redundant, to the extent it's even
accurate -- this is really just the URL we want to display,
which may or may not involve a subdomain.  Similarly "external".

The former `external_api_path_subdomain` was never a path -- it's a
host, followed by a path, which together form a scheme-relative URL.
I'm not quite convinced that value is actually the right thing in
2 of the 3 places we use it, but fixing that can start by giving an
accurate name to the thing we have.
2017-10-30 18:29:29 -07:00
rht c4fcff7178 refactor: Replace super(.*self) with Python 3-specific super().
We change all the instances except for the `test_helpers.py`
TimeTrackingCursor monkey-patching, which actually needs to specify
the base class.
2017-10-30 14:30:25 -07:00
Brock Whittaker 8c9579eb86 webpack.assets.json: Add a custom JS bundle for /api.
Note that we also need the JS bundle for /help because that
includes the JS code for our sidebar dropdowns.
2017-10-29 18:18:32 -07:00
Harshit Bansal 1067cfd92a node_modules: Add npm packages for the supported emojisets.
This commit adds emoji datasource packages for the rest of the emojiset
packages which we will use for serving images in notifications.
2017-10-28 10:38:32 -07:00
Tim Abbott 0d1194811f mypy: Remove ignores for a few typeshed bugs fixed upstream. 2017-10-27 17:09:00 -07:00
Tim Abbott 8d00111a27 vagrant: Skip the shell virtualenv warning on provision. 2017-10-27 11:44:47 -07:00
rht e296841447 tools/documentation_crawler: Use python 3 syntax for typing. 2017-10-26 21:58:22 -07:00
Tim Abbott 8e2cdedf9a lint: Fix lines in Python codebase longer than 120 characters. 2017-10-26 17:47:30 -07:00
Tim Abbott b936e8c24b lint: Fix lines in Python codebase longer than 125 characters. 2017-10-26 17:36:54 -07:00
Tim Abbott be619fe881 lint: Wrap many very long lines in the Python codebase.
This decreases the maximum line length in our Python codebase to 130.
2017-10-26 17:31:58 -07:00
Greg Price 4a6e867046 lint: Disable a buggy `pep8` rule on line breaks and operators.
I'd much rather see something like

    if (thing_is_permissible(user, thing)
            or (user_possesses_hammer(user)
                and glass_break_requested(thing))):

than

    if (thing_is_permissible(user, thing) or
            (user_possesses_hammer(user) and
                glass_break_requested(thing))):

because the former makes the overall logic much easier to scan.
Similarly for a formula full of arithmetic rather than Boolean
operators.  And the actual PEP 8 agrees (though until 2016 it
unfortunately had the opposite advice.)

The upstream linter still applies the backward rule, so disable that.
2017-10-26 10:29:17 -07:00
Brock Whittaker a9a9333b2a portico: Show whether the user is logged in.
This creates a dropdown in place of the normal register/login links
you get when logged out, with an option to go to the app or log out if
that appears you click on the avatar.

A bit more work is needed to make this look really good, but it's a
great start.
2017-10-25 14:59:24 -07:00
Aditya Bansal 0ddad98c98 features.html: Cleanup to use 4 space and consistant indentation. 2017-10-25 10:58:57 -07:00
Aditya Bansal 3fb0f7c092 home.html: Cleanup to use 4 space indentation.
Tweaked by tabbott to also make the translation tags make more sense.
2017-10-25 10:58:49 -07:00
Tim Abbott 618ace1f5c build-release-tarball: Fix missing CSS-inlined email templates.
These apparently we're being included.
2017-10-20 14:08:25 -07:00
Tim Abbott c9e29b1c54 check-templates: Document status of remaining bad_files items. 2017-10-19 16:56:46 -07:00